1996-07-09 08:22:35 +02:00
/*-------------------------------------------------------------------------
*
1999-02-14 00:22:53 +01:00
* analyze . c
2007-03-13 01:33:44 +01:00
* transform the raw parse tree into a query tree
*
* For optimizable statements , we are careful to obtain a suitable lock on
* each referenced table , and other modules of the backend preserve or
* re - obtain these locks before depending on the results . It is therefore
* okay to do significant semantic analysis of these statements . For
* utility commands , no locks are obtained here ( and if they were , we could
* not be sure we ' d still have them at execution ) . Hence the general rule
* for utility commands is to just dump them into a Query node untransformed .
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
* DECLARE CURSOR , EXPLAIN , and CREATE TABLE AS are exceptions because they
* contain optimizable statements , which we should transform .
2007-03-13 01:33:44 +01:00
*
1996-07-09 08:22:35 +02:00
*
2021-01-02 19:06:25 +01:00
* Portions Copyright ( c ) 1996 - 2021 , PostgreSQL Global Development Group
2000-01-26 06:58:53 +01:00
* Portions Copyright ( c ) 1994 , Regents of the University of California
1996-07-09 08:22:35 +02:00
*
2010-09-20 22:08:53 +02:00
* src / backend / parser / analyze . c
1996-07-09 08:22:35 +02:00
*
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
*/
1997-11-25 23:07:18 +01:00
1997-11-26 02:14:33 +01:00
# include "postgres.h"
1999-07-16 07:00:38 +02:00
2009-01-22 21:16:10 +01:00
# include "access/sysattr.h"
Reconsider the handling of procedure OUT parameters.
Commit 2453ea142 redefined pg_proc.proargtypes to include the types of
OUT parameters, for procedures only. While that had some advantages
for implementing the SQL-spec behavior of DROP PROCEDURE, it was pretty
disastrous from a number of other perspectives. Notably, since the
primary key of pg_proc is name + proargtypes, this made it possible to
have multiple procedures with identical names + input arguments and
differing output argument types. That would make it impossible to call
any one of the procedures by writing just NULL (or "?", or any other
data-type-free notation) for the output argument(s). The change also
seems likely to cause grave confusion for client applications that
examine pg_proc and expect the traditional definition of proargtypes.
Hence, revert the definition of proargtypes to what it was, and
undo a number of complications that had been added to support that.
To support the SQL-spec behavior of DROP PROCEDURE, when there are
no argmode markers in the command's parameter list, we perform the
lookup both ways (that is, matching against both proargtypes and
proallargtypes), succeeding if we get just one unique match.
In principle this could result in ambiguous-function failures
that would not happen when using only one of the two rules.
However, overloading of procedure names is thought to be a pretty
rare usage, so this shouldn't cause many problems in practice.
Postgres-specific code such as pg_dump can defend against any
possibility of such failures by being careful to specify argmodes
for all procedure arguments.
This also fixes a few other bugs in the area of CALL statements
with named parameters, and improves the documentation a little.
catversion bump forced because the representation of procedures
with OUT arguments changes.
Discussion: https://postgr.es/m/3742981.1621533210@sss.pgh.pa.us
2021-06-10 23:11:36 +02:00
# include "catalog/pg_proc.h"
1999-07-16 07:00:38 +02:00
# include "catalog/pg_type.h"
2012-11-12 01:56:10 +01:00
# include "miscadmin.h"
1997-11-25 23:07:18 +01:00
# include "nodes/makefuncs.h"
2008-08-26 00:42:34 +02:00
# include "nodes/nodeFuncs.h"
2019-01-29 21:48:51 +01:00
# include "optimizer/optimizer.h"
1997-11-25 23:07:18 +01:00
# include "parser/analyze.h"
# include "parser/parse_agg.h"
1997-11-26 02:14:33 +01:00
# include "parser/parse_clause.h"
2000-10-05 21:11:39 +02:00
# include "parser/parse_coerce.h"
2011-03-20 01:29:08 +01:00
# include "parser/parse_collate.h"
2008-10-04 23:56:55 +02:00
# include "parser/parse_cte.h"
2018-02-21 00:03:31 +01:00
# include "parser/parse_expr.h"
# include "parser/parse_func.h"
2008-08-07 03:11:52 +02:00
# include "parser/parse_oper.h"
2009-10-31 02:41:31 +01:00
# include "parser/parse_param.h"
1997-11-25 23:07:18 +01:00
# include "parser/parse_relation.h"
# include "parser/parse_target.h"
2021-01-04 17:52:00 +01:00
# include "parser/parse_type.h"
2006-07-11 18:35:33 +02:00
# include "parser/parsetree.h"
2008-09-01 22:42:46 +02:00
# include "rewrite/rewriteManip.h"
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
# include "utils/backend_status.h"
2021-01-04 17:52:00 +01:00
# include "utils/builtins.h"
2021-04-07 19:06:47 +02:00
# include "utils/guc.h"
# include "utils/queryjumble.h"
2008-06-19 02:46:06 +02:00
# include "utils/rel.h"
Reconsider the handling of procedure OUT parameters.
Commit 2453ea142 redefined pg_proc.proargtypes to include the types of
OUT parameters, for procedures only. While that had some advantages
for implementing the SQL-spec behavior of DROP PROCEDURE, it was pretty
disastrous from a number of other perspectives. Notably, since the
primary key of pg_proc is name + proargtypes, this made it possible to
have multiple procedures with identical names + input arguments and
differing output argument types. That would make it impossible to call
any one of the procedures by writing just NULL (or "?", or any other
data-type-free notation) for the output argument(s). The change also
seems likely to cause grave confusion for client applications that
examine pg_proc and expect the traditional definition of proargtypes.
Hence, revert the definition of proargtypes to what it was, and
undo a number of complications that had been added to support that.
To support the SQL-spec behavior of DROP PROCEDURE, when there are
no argmode markers in the command's parameter list, we perform the
lookup both ways (that is, matching against both proargtypes and
proallargtypes), succeeding if we get just one unique match.
In principle this could result in ambiguous-function failures
that would not happen when using only one of the two rules.
However, overloading of procedure names is thought to be a pretty
rare usage, so this shouldn't cause many problems in practice.
Postgres-specific code such as pg_dump can defend against any
possibility of such failures by being careful to specify argmodes
for all procedure arguments.
This also fixes a few other bugs in the area of CALL statements
with named parameters, and improves the documentation a little.
catversion bump forced because the representation of procedures
with OUT arguments changes.
Discussion: https://postgr.es/m/3742981.1621533210@sss.pgh.pa.us
2021-06-10 23:11:36 +02:00
# include "utils/syscache.h"
2000-08-22 14:59:04 +02:00
2001-10-12 02:07:15 +02:00
2012-03-27 21:14:13 +02:00
/* Hook for plugins to get control at end of parse analysis */
post_parse_analyze_hook_type post_parse_analyze_hook = NULL ;
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
static Query * transformOptionalSelectInto ( ParseState * pstate , Node * parseTree ) ;
1996-07-09 08:22:35 +02:00
static Query * transformDeleteStmt ( ParseState * pstate , DeleteStmt * stmt ) ;
2007-06-24 00:12:52 +02:00
static Query * transformInsertStmt ( ParseState * pstate , InsertStmt * stmt ) ;
2018-04-12 12:22:56 +02:00
static List * transformInsertRow ( ParseState * pstate , List * exprlist ,
List * stmtcols , List * icolumns , List * attrnos ,
bool strip_indirection ) ;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
static OnConflictExpr * transformOnConflictClause ( ParseState * pstate ,
OnConflictClause * onConflictClause ) ;
2010-09-18 20:37:01 +02:00
static int count_rowexpr_columns ( ParseState * pstate , Node * expr ) ;
1998-01-09 21:06:08 +01:00
static Query * transformSelectStmt ( ParseState * pstate , SelectStmt * stmt ) ;
2006-08-02 03:59:48 +02:00
static Query * transformValuesClause ( ParseState * pstate , SelectStmt * stmt ) ;
2000-11-05 01:15:54 +01:00
static Query * transformSetOperationStmt ( ParseState * pstate , SelectStmt * stmt ) ;
2008-08-29 01:09:48 +02:00
static Node * transformSetOperationTree ( ParseState * pstate , SelectStmt * stmt ,
2011-03-16 02:51:36 +01:00
bool isTopLevel , List * * targetlist ) ;
2009-09-09 05:32:52 +02:00
static void determineRecursiveColTypes ( ParseState * pstate ,
2011-03-16 02:51:36 +01:00
Node * larg , List * nrtargetlist ) ;
2021-04-07 21:30:08 +02:00
static Query * transformReturnStmt ( ParseState * pstate , ReturnStmt * stmt ) ;
1998-01-09 21:06:08 +01:00
static Query * transformUpdateStmt ( ParseState * pstate , UpdateStmt * stmt ) ;
2007-06-24 00:12:52 +02:00
static List * transformReturningList ( ParseState * pstate , List * returningList ) ;
2018-04-12 12:22:56 +02:00
static List * transformUpdateTargetList ( ParseState * pstate ,
List * targetList ) ;
2021-01-04 17:52:00 +01:00
static Query * transformPLAssignStmt ( ParseState * pstate ,
PLAssignStmt * stmt ) ;
2007-04-28 00:05:49 +02:00
static Query * transformDeclareCursorStmt ( ParseState * pstate ,
DeclareCursorStmt * stmt ) ;
static Query * transformExplainStmt ( ParseState * pstate ,
ExplainStmt * stmt ) ;
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
static Query * transformCreateTableAsStmt ( ParseState * pstate ,
CreateTableAsStmt * stmt ) ;
2018-02-21 00:03:31 +01:00
static Query * transformCallStmt ( ParseState * pstate ,
CallStmt * stmt ) ;
2009-10-28 15:55:47 +01:00
static void transformLockingClause ( ParseState * pstate , Query * qry ,
LockingClause * lc , bool pushedDown ) ;
2016-05-24 01:08:26 +02:00
# ifdef RAW_EXPRESSION_COVERAGE_TEST
static bool test_raw_expression_coverage ( Node * node , void * context ) ;
# endif
2000-09-12 23:07:18 +02:00
1999-07-19 02:26:20 +02:00
1996-07-09 08:22:35 +02:00
/*
2003-04-30 00:13:11 +02:00
* parse_analyze
* Analyze a raw parse tree and transform it to Query form .
*
* Optionally , information about $ n parameter types can be supplied .
* References to $ n indexes not defined by paramTypes [ ] are disallowed .
1996-07-09 08:22:35 +02:00
*
2007-06-24 00:12:52 +02:00
* The result is a Query node . Optimizable statements require considerable
* transformation , while utility - type statements are simply hung off
2000-10-07 02:58:23 +02:00
* a dummy CMD_UTILITY Query node .
1996-07-09 08:22:35 +02:00
*/
2007-06-24 00:12:52 +02:00
Query *
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
parse_analyze ( RawStmt * parseTree , const char * sourceText ,
2017-04-01 06:17:18 +02:00
Oid * paramTypes , int numParams ,
QueryEnvironment * queryEnv )
2003-04-30 00:13:11 +02:00
{
ParseState * pstate = make_parsestate ( NULL ) ;
2007-06-24 00:12:52 +02:00
Query * query ;
2021-04-07 19:06:47 +02:00
JumbleState * jstate = NULL ;
2003-04-30 00:13:11 +02:00
Adjust things so that the query_string of a cached plan and the sourceText of
a portal are never NULL, but reliably provide the source text of the query.
It turns out that there was only one place that was really taking a short-cut,
which was the 'EXECUTE' utility statement. That doesn't seem like a
sufficiently critical performance hotspot to justify not offering a guarantee
of validity of the portal source text. Fix it to copy the source text over
from the cached plan. Add Asserts in the places that set up cached plans and
portals to reject null source strings, and simplify a bunch of places that
formerly needed to guard against nulls.
There may be a few places that cons up statements for execution without
having any source text at all; I found one such in ConvertTriggerToFK().
It seems sufficient to inject a phony source string in such a case,
for instance
ProcessUtility((Node *) atstmt,
"(generated ALTER TABLE ADD FOREIGN KEY command)",
NULL, false, None_Receiver, NULL);
We should take a second look at the usage of debug_query_string,
particularly the recently added current_query() SQL function.
ITAGAKI Takahiro and Tom Lane
2008-07-18 22:26:06 +02:00
Assert ( sourceText ! = NULL ) ; /* required as of 8.4 */
2006-03-14 23:48:25 +01:00
pstate - > p_sourcetext = sourceText ;
2009-10-31 02:41:31 +01:00
if ( numParams > 0 )
parse_fixed_parameters ( pstate , paramTypes , numParams ) ;
2003-04-30 00:13:11 +02:00
2017-04-01 06:17:18 +02:00
pstate - > p_queryEnv = queryEnv ;
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
query = transformTopLevelStmt ( pstate , parseTree ) ;
2003-04-30 00:13:11 +02:00
2021-05-15 20:13:09 +02:00
if ( IsQueryIdEnabled ( ) )
2021-04-07 19:06:47 +02:00
jstate = JumbleQuery ( query , sourceText ) ;
2012-03-27 21:14:13 +02:00
if ( post_parse_analyze_hook )
2021-04-07 19:06:47 +02:00
( * post_parse_analyze_hook ) ( pstate , query , jstate ) ;
2012-03-27 21:14:13 +02:00
2007-06-24 00:12:52 +02:00
free_parsestate ( pstate ) ;
2003-04-30 00:13:11 +02:00
2021-04-20 18:22:26 +02:00
pgstat_report_query_id ( query - > queryId , false ) ;
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
2007-06-24 00:12:52 +02:00
return query ;
2003-04-30 00:13:11 +02:00
}
/*
* parse_analyze_varparams
*
* This variant is used when it ' s okay to deduce information about $ n
* symbol datatypes from context . The passed - in paramTypes [ ] array can
* be modified or enlarged ( via repalloc ) .
*/
2007-06-24 00:12:52 +02:00
Query *
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
parse_analyze_varparams ( RawStmt * parseTree , const char * sourceText ,
2006-03-14 23:48:25 +01:00
Oid * * paramTypes , int * numParams )
2003-04-30 00:13:11 +02:00
{
ParseState * pstate = make_parsestate ( NULL ) ;
2007-06-24 00:12:52 +02:00
Query * query ;
2021-04-07 19:06:47 +02:00
JumbleState * jstate = NULL ;
2003-04-30 00:13:11 +02:00
Adjust things so that the query_string of a cached plan and the sourceText of
a portal are never NULL, but reliably provide the source text of the query.
It turns out that there was only one place that was really taking a short-cut,
which was the 'EXECUTE' utility statement. That doesn't seem like a
sufficiently critical performance hotspot to justify not offering a guarantee
of validity of the portal source text. Fix it to copy the source text over
from the cached plan. Add Asserts in the places that set up cached plans and
portals to reject null source strings, and simplify a bunch of places that
formerly needed to guard against nulls.
There may be a few places that cons up statements for execution without
having any source text at all; I found one such in ConvertTriggerToFK().
It seems sufficient to inject a phony source string in such a case,
for instance
ProcessUtility((Node *) atstmt,
"(generated ALTER TABLE ADD FOREIGN KEY command)",
NULL, false, None_Receiver, NULL);
We should take a second look at the usage of debug_query_string,
particularly the recently added current_query() SQL function.
ITAGAKI Takahiro and Tom Lane
2008-07-18 22:26:06 +02:00
Assert ( sourceText ! = NULL ) ; /* required as of 8.4 */
2006-03-14 23:48:25 +01:00
pstate - > p_sourcetext = sourceText ;
2009-10-31 02:41:31 +01:00
parse_variable_parameters ( pstate , paramTypes , numParams ) ;
2003-04-30 00:13:11 +02:00
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
query = transformTopLevelStmt ( pstate , parseTree ) ;
2003-04-30 00:13:11 +02:00
2008-09-01 22:42:46 +02:00
/* make sure all is well with parameter types */
2009-10-31 02:41:31 +01:00
check_variable_parameters ( pstate , query ) ;
2003-04-30 00:13:11 +02:00
2021-05-15 20:13:09 +02:00
if ( IsQueryIdEnabled ( ) )
2021-04-07 19:06:47 +02:00
jstate = JumbleQuery ( query , sourceText ) ;
2012-03-27 21:14:13 +02:00
if ( post_parse_analyze_hook )
2021-04-07 19:06:47 +02:00
( * post_parse_analyze_hook ) ( pstate , query , jstate ) ;
2012-03-27 21:14:13 +02:00
2007-06-24 00:12:52 +02:00
free_parsestate ( pstate ) ;
2003-04-30 00:13:11 +02:00
2021-04-20 18:22:26 +02:00
pgstat_report_query_id ( query - > queryId , false ) ;
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
2007-06-24 00:12:52 +02:00
return query ;
2003-04-30 00:13:11 +02:00
}
/*
* parse_sub_analyze
* Entry point for recursively analyzing a sub - statement .
*/
2007-06-24 00:12:52 +02:00
Query *
2009-09-09 05:32:52 +02:00
parse_sub_analyze ( Node * parseTree , ParseState * parentParseState ,
2009-10-27 18:11:18 +01:00
CommonTableExpr * parentCTE ,
Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column). This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on. For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison". Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.
To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command. In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.
Also, change creation of an unknown-type column in a relation from a
warning to a hard error. The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE. We want to forbid that because
it's nothing but a foot-gun.
This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have. Add a checking pass to detect that. While at it,
we can detect tables or composite types that would fail, essentially
for free. Those would fail safely anyway later on, but we might as
well fail earlier.
This patch is by me, but it owes something to previous investigations
by Rahila Syed. Also thanks to Ashutosh Bapat and Michael Paquier for
review.
Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 15:17:18 +01:00
bool locked_from_parent ,
bool resolve_unknowns )
1996-07-09 08:22:35 +02:00
{
2000-10-07 02:58:23 +02:00
ParseState * pstate = make_parsestate ( parentParseState ) ;
2002-02-26 23:47:12 +01:00
Query * query ;
2003-04-30 00:13:11 +02:00
2009-09-09 05:32:52 +02:00
pstate - > p_parent_cte = parentCTE ;
2009-10-27 18:11:18 +01:00
pstate - > p_locked_from_parent = locked_from_parent ;
Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column). This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on. For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison". Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.
To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command. In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.
Also, change creation of an unknown-type column in a relation from a
warning to a hard error. The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE. We want to forbid that because
it's nothing but a foot-gun.
This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have. Add a checking pass to detect that. While at it,
we can detect tables or composite types that would fail, essentially
for free. Those would fail safely anyway later on, but we might as
well fail earlier.
This patch is by me, but it owes something to previous investigations
by Rahila Syed. Also thanks to Ashutosh Bapat and Michael Paquier for
review.
Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 15:17:18 +01:00
pstate - > p_resolve_unknowns = resolve_unknowns ;
2009-09-09 05:32:52 +02:00
2007-06-24 00:12:52 +02:00
query = transformStmt ( pstate , parseTree ) ;
1999-07-19 02:26:20 +02:00
2007-06-24 00:12:52 +02:00
free_parsestate ( pstate ) ;
1998-01-20 06:05:08 +01:00
2007-06-24 00:12:52 +02:00
return query ;
2000-09-12 23:07:18 +02:00
}
1996-07-09 08:22:35 +02:00
/*
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
* transformTopLevelStmt -
2000-10-07 02:58:23 +02:00
* transform a Parse tree into a Query tree .
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
*
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
* This function is just responsible for transferring statement location data
* from the RawStmt into the finished Query .
*/
Query *
transformTopLevelStmt ( ParseState * pstate , RawStmt * parseTree )
{
Query * result ;
/* We're at top level, so allow SELECT INTO */
result = transformOptionalSelectInto ( pstate , parseTree - > stmt ) ;
result - > stmt_location = parseTree - > stmt_location ;
result - > stmt_len = parseTree - > stmt_len ;
return result ;
}
/*
* transformOptionalSelectInto -
* If SELECT has INTO , convert it to CREATE TABLE AS .
*
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
* The only thing we do here that we don ' t do in transformStmt ( ) is to
* convert SELECT . . . INTO into CREATE TABLE AS . Since utility statements
* aren ' t allowed within larger statements , this is only allowed at the top
* of the parse tree , and so we only try it before entering the recursive
* transformStmt ( ) processing .
*/
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
static Query *
transformOptionalSelectInto ( ParseState * pstate , Node * parseTree )
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
{
if ( IsA ( parseTree , SelectStmt ) )
{
SelectStmt * stmt = ( SelectStmt * ) parseTree ;
/* If it's a set-operation tree, drill down to leftmost SelectStmt */
while ( stmt & & stmt - > op ! = SETOP_NONE )
stmt = stmt - > larg ;
Assert ( stmt & & IsA ( stmt , SelectStmt ) & & stmt - > larg = = NULL ) ;
if ( stmt - > intoClause )
{
CreateTableAsStmt * ctas = makeNode ( CreateTableAsStmt ) ;
ctas - > query = parseTree ;
ctas - > into = stmt - > intoClause ;
2020-07-11 06:32:28 +02:00
ctas - > objtype = OBJECT_TABLE ;
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
ctas - > is_select_into = true ;
/*
* Remove the intoClause from the SelectStmt . This makes it safe
* for transformSelectStmt to complain if it finds intoClause set
* ( implying that the INTO appeared in a disallowed place ) .
*/
stmt - > intoClause = NULL ;
parseTree = ( Node * ) ctas ;
}
}
return transformStmt ( pstate , parseTree ) ;
}
/*
* transformStmt -
* recursively transform a Parse tree into a Query tree .
1996-07-09 08:22:35 +02:00
*/
2007-06-24 00:12:52 +02:00
Query *
transformStmt ( ParseState * pstate , Node * parseTree )
1996-07-09 08:22:35 +02:00
{
2007-06-24 00:12:52 +02:00
Query * result ;
1997-09-07 07:04:48 +02:00
2016-05-24 01:08:26 +02:00
/*
* We apply RAW_EXPRESSION_COVERAGE_TEST testing to basic DML statements ;
* we can ' t just run it on everything because raw_expression_tree_walker ( )
* doesn ' t claim to handle utility statements .
*/
# ifdef RAW_EXPRESSION_COVERAGE_TEST
switch ( nodeTag ( parseTree ) )
{
case T_SelectStmt :
case T_InsertStmt :
case T_UpdateStmt :
case T_DeleteStmt :
( void ) test_raw_expression_coverage ( parseTree , NULL ) ;
break ;
default :
break ;
}
# endif /* RAW_EXPRESSION_COVERAGE_TEST */
1996-07-09 08:22:35 +02:00
switch ( nodeTag ( parseTree ) )
{
1998-01-20 06:05:08 +01:00
/*
* Optimizable statements
*/
1998-01-09 21:06:08 +01:00
case T_InsertStmt :
2007-06-24 00:12:52 +02:00
result = transformInsertStmt ( pstate , ( InsertStmt * ) parseTree ) ;
1996-07-09 08:22:35 +02:00
break ;
1997-09-07 07:04:48 +02:00
1996-07-09 08:22:35 +02:00
case T_DeleteStmt :
result = transformDeleteStmt ( pstate , ( DeleteStmt * ) parseTree ) ;
break ;
1997-09-07 07:04:48 +02:00
1998-01-09 21:06:08 +01:00
case T_UpdateStmt :
result = transformUpdateStmt ( pstate , ( UpdateStmt * ) parseTree ) ;
1996-07-09 08:22:35 +02:00
break ;
1997-09-07 07:04:48 +02:00
1998-01-09 21:06:08 +01:00
case T_SelectStmt :
2006-08-02 03:59:48 +02:00
{
SelectStmt * n = ( SelectStmt * ) parseTree ;
if ( n - > valuesLists )
result = transformValuesClause ( pstate , n ) ;
else if ( n - > op = = SETOP_NONE )
result = transformSelectStmt ( pstate , n ) ;
else
result = transformSetOperationStmt ( pstate , n ) ;
}
1996-07-09 08:22:35 +02:00
break ;
1997-09-07 07:04:48 +02:00
2021-04-07 21:30:08 +02:00
case T_ReturnStmt :
result = transformReturnStmt ( pstate , ( ReturnStmt * ) parseTree ) ;
break ;
2021-01-04 17:52:00 +01:00
case T_PLAssignStmt :
result = transformPLAssignStmt ( pstate ,
( PLAssignStmt * ) parseTree ) ;
break ;
2007-04-28 00:05:49 +02:00
/*
* Special cases
*/
case T_DeclareCursorStmt :
result = transformDeclareCursorStmt ( pstate ,
( DeclareCursorStmt * ) parseTree ) ;
break ;
case T_ExplainStmt :
result = transformExplainStmt ( pstate ,
( ExplainStmt * ) parseTree ) ;
break ;
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
case T_CreateTableAsStmt :
result = transformCreateTableAsStmt ( pstate ,
( CreateTableAsStmt * ) parseTree ) ;
break ;
2018-02-21 00:03:31 +01:00
case T_CallStmt :
result = transformCallStmt ( pstate ,
( CallStmt * ) parseTree ) ;
2018-03-20 00:45:25 +01:00
break ;
2018-02-21 00:03:31 +01:00
1996-07-09 08:22:35 +02:00
default :
1997-09-07 07:04:48 +02:00
1996-07-09 08:22:35 +02:00
/*
2007-03-13 01:33:44 +01:00
* other statements don ' t require any transformation ; just return
* the original parsetree with a Query node plastered on top .
1996-07-09 08:22:35 +02:00
*/
result = makeNode ( Query ) ;
result - > commandType = CMD_UTILITY ;
result - > utilityStmt = ( Node * ) parseTree ;
break ;
}
2003-05-02 22:54:36 +02:00
/* Mark as original query until we learn differently */
result - > querySource = QSRC_ORIGINAL ;
result - > canSetTag = true ;
1996-07-09 08:22:35 +02:00
return result ;
}
2008-12-13 03:00:20 +01:00
/*
* analyze_requires_snapshot
* Returns true if a snapshot must be set before doing parse analysis
* on the given raw parse tree .
*
2014-11-12 21:58:37 +01:00
* Classification here should match transformStmt ( ) .
2008-12-13 03:00:20 +01:00
*/
bool
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
analyze_requires_snapshot ( RawStmt * parseTree )
2008-12-13 03:00:20 +01:00
{
bool result ;
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
switch ( nodeTag ( parseTree - > stmt ) )
2008-12-13 03:00:20 +01:00
{
/*
* Optimizable statements
*/
case T_InsertStmt :
case T_DeleteStmt :
case T_UpdateStmt :
case T_SelectStmt :
2021-01-04 17:52:00 +01:00
case T_PLAssignStmt :
2008-12-13 03:00:20 +01:00
result = true ;
break ;
/*
* Special cases
*/
case T_DeclareCursorStmt :
case T_ExplainStmt :
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
case T_CreateTableAsStmt :
2010-01-15 23:36:35 +01:00
/* yes, because we must analyze the contained statement */
2008-12-13 03:00:20 +01:00
result = true ;
break ;
default :
2010-01-15 23:36:35 +01:00
/* other utility statements don't have any real parse analysis */
2008-12-13 03:00:20 +01:00
result = false ;
break ;
}
return result ;
}
1996-07-09 08:22:35 +02:00
/*
* transformDeleteStmt -
* transforms a Delete Statement
*/
static Query *
transformDeleteStmt ( ParseState * pstate , DeleteStmt * stmt )
{
Query * qry = makeNode ( Query ) ;
2014-01-07 21:25:16 +01:00
ParseNamespaceItem * nsitem ;
2000-09-29 20:21:41 +02:00
Node * qual ;
1996-07-09 08:22:35 +02:00
qry - > commandType = CMD_DELETE ;
2010-10-16 01:53:59 +02:00
/* process the WITH clause independently of all else */
if ( stmt - > withClause )
{
qry - > hasRecursive = stmt - > withClause - > recursive ;
qry - > cteList = transformWithClause ( pstate , stmt - > withClause ) ;
2011-02-26 00:56:23 +01:00
qry - > hasModifyingCTE = pstate - > p_hasModifyingCTE ;
2010-10-16 01:53:59 +02:00
}
2001-02-14 22:35:07 +01:00
/* set up range table with just the result rel */
2002-03-22 03:56:37 +01:00
qry - > resultRelation = setTargetTable ( pstate , stmt - > relation ,
2016-12-23 19:35:11 +01:00
stmt - > relation - > inh ,
2004-01-15 00:01:55 +01:00
true ,
ACL_DELETE ) ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
nsitem = pstate - > p_target_nsitem ;
2014-01-07 21:25:16 +01:00
/* there's no DISTINCT in DELETE */
2000-01-27 19:11:50 +01:00
qry - > distinctClause = NIL ;
1996-07-09 08:22:35 +02:00
2014-01-12 01:03:12 +01:00
/* subqueries in USING cannot access the result relation */
2014-01-07 21:25:16 +01:00
nsitem - > p_lateral_only = true ;
2014-01-12 01:03:12 +01:00
nsitem - > p_lateral_ok = false ;
2014-01-07 21:25:16 +01:00
2005-04-07 03:51:41 +02:00
/*
* The USING clause is non - standard SQL syntax , and is equivalent in
* functionality to the FROM list that can be specified for UPDATE . The
* USING keyword is used rather than FROM because FROM is already a
* keyword in the DELETE syntax .
*/
transformFromClause ( pstate , stmt - > usingClause ) ;
2014-01-12 01:03:12 +01:00
/* remaining clauses can reference the result relation normally */
2014-01-07 21:25:16 +01:00
nsitem - > p_lateral_only = false ;
2014-01-12 01:03:12 +01:00
nsitem - > p_lateral_ok = true ;
2014-01-07 21:25:16 +01:00
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
qual = transformWhereClause ( pstate , stmt - > whereClause ,
EXPR_KIND_WHERE , " WHERE " ) ;
1998-02-26 05:46:47 +01:00
2006-08-12 04:52:06 +02:00
qry - > returningList = transformReturningList ( pstate , stmt - > returningList ) ;
2000-09-29 20:21:41 +02:00
/* done building the range table and jointree */
1996-07-09 08:22:35 +02:00
qry - > rtable = pstate - > p_rtable ;
2000-09-29 20:21:41 +02:00
qry - > jointree = makeFromExpr ( pstate - > p_joinlist , qual ) ;
1997-09-07 07:04:48 +02:00
1999-11-15 03:00:15 +01:00
qry - > hasSubLinks = pstate - > p_hasSubLinks ;
2008-12-28 19:54:01 +01:00
qry - > hasWindowFuncs = pstate - > p_hasWindowFuncs ;
2016-09-13 19:54:24 +02:00
qry - > hasTargetSRFs = pstate - > p_hasTargetSRFs ;
2012-02-08 19:15:02 +01:00
qry - > hasAggs = pstate - > p_hasAggs ;
1996-07-09 08:22:35 +02:00
2011-03-20 01:29:08 +01:00
assign_query_collations ( pstate , qry ) ;
2019-01-17 06:33:01 +01:00
/* this must be done after collations, for reliable comparison of exprs */
if ( pstate - > p_hasAggs )
parseCheckAggregates ( pstate , qry ) ;
2000-10-05 21:11:39 +02:00
return qry ;
1996-07-09 08:22:35 +02:00
}
/*
* transformInsertStmt -
* transform an Insert Statement
*/
static Query *
2007-06-24 00:12:52 +02:00
transformInsertStmt ( ParseState * pstate , InsertStmt * stmt )
1996-07-09 08:22:35 +02:00
{
1999-07-19 02:26:20 +02:00
Query * qry = makeNode ( Query ) ;
2006-08-02 03:59:48 +02:00
SelectStmt * selectStmt = ( SelectStmt * ) stmt - > selectStmt ;
List * exprList = NIL ;
bool isGeneralSelect ;
2001-02-14 22:35:07 +01:00
List * sub_rtable ;
2012-08-08 22:41:04 +02:00
List * sub_namespace ;
1997-10-12 09:09:20 +02:00
List * icolumns ;
1999-11-01 06:06:21 +01:00
List * attrnos ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
ParseNamespaceItem * nsitem ;
2006-08-02 03:59:48 +02:00
RangeTblEntry * rte ;
2004-05-26 06:41:50 +02:00
ListCell * icols ;
ListCell * attnos ;
2006-08-02 03:59:48 +02:00
ListCell * lc ;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
bool isOnConflictUpdate ;
AclMode targetPerms ;
1996-07-09 08:22:35 +02:00
2010-10-16 01:53:59 +02:00
/* There can't be any outer WITH to worry about */
Assert ( pstate - > p_ctenamespace = = NIL ) ;
1996-07-09 08:22:35 +02:00
qry - > commandType = CMD_INSERT ;
1996-10-30 03:02:41 +01:00
pstate - > p_is_insert = true ;
1996-07-09 08:22:35 +02:00
2010-10-16 01:53:59 +02:00
/* process the WITH clause independently of all else */
if ( stmt - > withClause )
{
qry - > hasRecursive = stmt - > withClause - > recursive ;
qry - > cteList = transformWithClause ( pstate , stmt - > withClause ) ;
2011-02-26 00:56:23 +01:00
qry - > hasModifyingCTE = pstate - > p_hasModifyingCTE ;
2010-10-16 01:53:59 +02:00
}
2017-04-06 14:33:16 +02:00
qry - > override = stmt - > override ;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
isOnConflictUpdate = ( stmt - > onConflictClause & &
stmt - > onConflictClause - > action = = ONCONFLICT_UPDATE ) ;
2006-08-02 03:59:48 +02:00
/*
* We have three cases to deal with : DEFAULT VALUES ( selectStmt = = NULL ) ,
* VALUES list , or general SELECT input . We special - case VALUES , both for
* efficiency and so we can handle DEFAULT specifications .
2010-10-03 02:02:27 +02:00
*
* The grammar allows attaching ORDER BY , LIMIT , FOR UPDATE , or WITH to a
* VALUES clause . If we have any of those , treat it as a general SELECT ;
* so it will work , but you can ' t use DEFAULT items together with those .
2006-08-02 03:59:48 +02:00
*/
2010-10-03 02:02:27 +02:00
isGeneralSelect = ( selectStmt & & ( selectStmt - > valuesLists = = NIL | |
selectStmt - > sortClause ! = NIL | |
selectStmt - > limitOffset ! = NULL | |
selectStmt - > limitCount ! = NULL | |
selectStmt - > lockingClause ! = NIL | |
selectStmt - > withClause ! = NULL ) ) ;
2006-08-02 03:59:48 +02:00
2000-11-08 23:10:03 +01:00
/*
2001-02-14 22:35:07 +01:00
* If a non - nil rangetable / namespace was passed in , and we are doing
* INSERT / SELECT , arrange to pass the rangetable / namespace down to the
* SELECT . This can only happen if we are inside a CREATE RULE , and in
* that case we want the rule ' s OLD and NEW rtable entries to appear as
* part of the SELECT ' s rtable , not as outer references for it . ( Kluge ! )
2005-06-05 02:38:11 +02:00
* The SELECT ' s joinlist is not affected however . We must do this before
2001-02-14 22:35:07 +01:00
* adding the target table to the INSERT ' s rtable .
*/
2006-08-02 03:59:48 +02:00
if ( isGeneralSelect )
2001-02-14 22:35:07 +01:00
{
sub_rtable = pstate - > p_rtable ;
pstate - > p_rtable = NIL ;
2012-08-08 22:41:04 +02:00
sub_namespace = pstate - > p_namespace ;
pstate - > p_namespace = NIL ;
2001-02-14 22:35:07 +01:00
}
else
{
sub_rtable = NIL ; /* not used, but keep compiler quiet */
2012-08-08 22:41:04 +02:00
sub_namespace = NIL ;
2001-02-14 22:35:07 +01:00
}
/*
* Must get write lock on INSERT target table before scanning SELECT , else
2000-11-08 23:10:03 +01:00
* we will grab the wrong kind of initial lock if the target table is also
2001-02-14 22:35:07 +01:00
* mentioned in the SELECT part . Note that the target table is not added
* to the joinlist or namespace .
2000-11-08 23:10:03 +01:00
*/
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
targetPerms = ACL_INSERT ;
if ( isOnConflictUpdate )
targetPerms | = ACL_UPDATE ;
2002-03-22 03:56:37 +01:00
qry - > resultRelation = setTargetTable ( pstate , stmt - > relation ,
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
false , false , targetPerms ) ;
2000-11-08 23:10:03 +01:00
2006-08-02 03:59:48 +02:00
/* Validate stmt->cols list, or build default list if no list given */
icolumns = checkInsertTargets ( pstate , stmt - > cols , & attrnos ) ;
Assert ( list_length ( icolumns ) = = list_length ( attrnos ) ) ;
1998-05-29 15:39:30 +02:00
/*
2006-08-02 03:59:48 +02:00
* Determine which variant of INSERT we have .
1999-01-21 17:08:55 +01:00
*/
2006-08-02 03:59:48 +02:00
if ( selectStmt = = NULL )
{
/*
* We have INSERT . . . DEFAULT VALUES . We can handle this case by
* emitting an empty targetlist - - - all columns will be defaulted when
* the planner expands the targetlist .
*/
exprList = NIL ;
}
else if ( isGeneralSelect )
2000-10-05 21:11:39 +02:00
{
2003-04-30 00:13:11 +02:00
/*
* We make the sub - pstate a child of the outer pstate so that it can
* see any Param definitions supplied from above . Since the outer
* pstate ' s rtable and namespace are presently empty , there are no
* side - effects of exposing names the sub - SELECT shouldn ' t be able to
* see .
*/
ParseState * sub_pstate = make_parsestate ( pstate ) ;
2006-08-02 03:59:48 +02:00
Query * selectQuery ;
1999-05-25 18:15:34 +02:00
2000-10-05 21:11:39 +02:00
/*
* Process the source SELECT .
*
* It is important that this be handled just like a standalone SELECT ;
* otherwise the behavior of SELECT within INSERT might be different
* from a stand - alone SELECT . ( Indeed , Postgres up through 6.5 had
* bugs of just that nature . . . )
Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column). This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on. For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison". Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.
To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command. In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.
Also, change creation of an unknown-type column in a relation from a
warning to a hard error. The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE. We want to forbid that because
it's nothing but a foot-gun.
This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have. Add a checking pass to detect that. While at it,
we can detect tables or composite types that would fail, essentially
for free. Those would fail safely anyway later on, but we might as
well fail earlier.
This patch is by me, but it owes something to previous investigations
by Rahila Syed. Also thanks to Ashutosh Bapat and Michael Paquier for
review.
Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 15:17:18 +01:00
*
* The sole exception is that we prevent resolving unknown - type
* outputs as TEXT . This does not change the semantics since if the
* column type matters semantically , it would have been resolved to
* something else anyway . Doing this lets us resolve such outputs as
* the target column ' s type , which we handle below .
2000-10-05 21:11:39 +02:00
*/
2001-02-14 22:35:07 +01:00
sub_pstate - > p_rtable = sub_rtable ;
2009-01-22 21:16:10 +01:00
sub_pstate - > p_joinexprs = NIL ; /* sub_rtable has no joins */
2012-08-08 22:41:04 +02:00
sub_pstate - > p_namespace = sub_namespace ;
Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column). This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on. For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison". Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.
To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command. In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.
Also, change creation of an unknown-type column in a relation from a
warning to a hard error. The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE. We want to forbid that because
it's nothing but a foot-gun.
This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have. Add a checking pass to detect that. While at it,
we can detect tables or composite types that would fail, essentially
for free. Those would fail safely anyway later on, but we might as
well fail earlier.
This patch is by me, but it owes something to previous investigations
by Rahila Syed. Also thanks to Ashutosh Bapat and Michael Paquier for
review.
Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 15:17:18 +01:00
sub_pstate - > p_resolve_unknowns = false ;
2001-02-14 22:35:07 +01:00
2007-06-24 00:12:52 +02:00
selectQuery = transformStmt ( sub_pstate , stmt - > selectStmt ) ;
2001-02-14 22:35:07 +01:00
2007-06-24 00:12:52 +02:00
free_parsestate ( sub_pstate ) ;
2000-10-05 21:11:39 +02:00
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
/* The grammar should have produced a SELECT */
2008-09-01 22:42:46 +02:00
if ( ! IsA ( selectQuery , Query ) | |
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
selectQuery - > commandType ! = CMD_SELECT )
2008-09-01 22:42:46 +02:00
elog ( ERROR , " unexpected non-SELECT command in INSERT ... SELECT " ) ;
2001-03-22 05:01:46 +01:00
2000-10-05 21:11:39 +02:00
/*
* Make the source be a subquery in the INSERT ' s rangetable , and add
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
* it to the INSERT ' s joinlist ( but not the namespace ) .
2000-10-05 21:11:39 +02:00
*/
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
nsitem = addRangeTableEntryForSubquery ( pstate ,
selectQuery ,
makeAlias ( " *SELECT* " , NIL ) ,
false ,
false ) ;
addNSItemToQuery ( pstate , nsitem , true , false , false ) ;
2001-03-22 05:01:46 +01:00
2005-02-19 20:33:08 +01:00
/*----------
2006-08-02 03:59:48 +02:00
* Generate an expression list for the INSERT that selects all the
* non - resjunk columns from the subquery . ( INSERT ' s tlist must be
2000-10-05 21:11:39 +02:00
* separate from the subquery ' s tlist because we may add columns ,
* insert datatype coercions , etc . )
*
2005-04-06 18:34:07 +02:00
* HACK : unknown - type constants and params in the SELECT ' s targetlist
2003-04-30 00:13:11 +02:00
* are copied up as - is rather than being referenced as subquery
* outputs . This is to ensure that when we try to coerce them to
* the target column ' s datatype , the right things happen ( see
2005-02-19 20:33:08 +01:00
* special cases in coerce_type ) . Otherwise , this fails :
* INSERT INTO foo SELECT ' bar ' , . . . FROM baz
* - - - - - - - - - -
2000-10-05 21:11:39 +02:00
*/
2006-08-02 03:59:48 +02:00
exprList = NIL ;
foreach ( lc , selectQuery - > targetList )
2000-10-05 21:11:39 +02:00
{
2006-08-02 03:59:48 +02:00
TargetEntry * tle = ( TargetEntry * ) lfirst ( lc ) ;
2002-12-12 16:49:42 +01:00
Expr * expr ;
2000-10-05 21:11:39 +02:00
2005-04-06 18:34:07 +02:00
if ( tle - > resjunk )
2000-10-05 21:11:39 +02:00
continue ;
2003-04-30 00:13:11 +02:00
if ( tle - > expr & &
2005-02-19 20:33:08 +01:00
( IsA ( tle - > expr , Const ) | | IsA ( tle - > expr , Param ) ) & &
2003-04-30 00:13:11 +02:00
exprType ( ( Node * ) tle - > expr ) = = UNKNOWNOID )
2000-10-05 21:11:39 +02:00
expr = tle - > expr ;
else
2008-10-07 03:47:55 +02:00
{
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
Var * var = makeVarFromTargetEntry ( nsitem - > p_rtindex , tle ) ;
2009-06-11 16:49:15 +02:00
2008-10-07 03:47:55 +02:00
var - > location = exprLocation ( ( Node * ) tle - > expr ) ;
expr = ( Expr * ) var ;
}
2006-08-02 03:59:48 +02:00
exprList = lappend ( exprList , expr ) ;
2000-10-05 21:11:39 +02:00
}
2006-08-02 03:59:48 +02:00
/* Prepare row for assignment to target table */
exprList = transformInsertRow ( pstate , exprList ,
stmt - > cols ,
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
icolumns , attrnos ,
false ) ;
2000-10-05 21:11:39 +02:00
}
2006-08-02 03:59:48 +02:00
else if ( list_length ( selectStmt - > valuesLists ) > 1 )
2000-10-05 21:11:39 +02:00
{
/*
2006-08-02 03:59:48 +02:00
* Process INSERT . . . VALUES with multiple VALUES sublists . We
* generate a VALUES RTE holding the transformed expression lists , and
* build up a targetlist containing Vars that reference the VALUES
* RTE .
2000-10-05 21:11:39 +02:00
*/
2006-08-02 03:59:48 +02:00
List * exprsLists = NIL ;
Fix reporting of column typmods for multi-row VALUES constructs.
expandRTE() and get_rte_attribute_type() reported the exprType() and
exprTypmod() values of the expressions in the first row of the VALUES as
being the column type/typmod returned by the VALUES RTE. That's fine for
the data type, since we coerce all expressions in a column to have the same
common type. But we don't coerce them to have a common typmod, so it was
possible for rows after the first one to return values that violate the
claimed column typmod. This leads to the incorrect result seen in bug
#14448 from Hassan Mahmood, as well as some other corner-case misbehaviors.
The desired behavior is the same as we use in other type-unification
cases: report the common typmod if there is one, but otherwise return -1
indicating no particular constraint. It's cheap for transformValuesClause
to determine the common typmod while transforming a multi-row VALUES, but
it'd be less cheap for expandRTE() and get_rte_attribute_type() to
re-determine that info every time they're asked --- possibly a lot less
cheap, if the VALUES has many rows. Therefore, the best fix is to record
the common typmods explicitly in a list in the VALUES RTE, as we were
already doing for column collations. This looks quite a bit like what
we're doing for CTE RTEs, so we can save a little bit of space and code by
unifying the representation for those two RTE types. They both now share
coltypes/coltypmods/colcollations fields. (At some point it might seem
desirable to populate those fields for all RTE types; but right now it
looks like constructing them for other RTE types would add more code and
cycles than it would save.)
The RTE change requires a catversion bump, so this fix is only usable
in HEAD. If we fix this at all in the back branches, the patch will
need to look quite different.
Report: https://postgr.es/m/20161205143037.4377.60754@wrigleys.postgresql.org
Discussion: https://postgr.es/m/27429.1480968538@sss.pgh.pa.us
2016-12-08 17:40:02 +01:00
List * coltypes = NIL ;
List * coltypmods = NIL ;
List * colcollations = NIL ;
2006-08-02 03:59:48 +02:00
int sublist_length = - 1 ;
2012-08-19 20:12:16 +02:00
bool lateral = false ;
2006-08-02 03:59:48 +02:00
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
Assert ( selectStmt - > intoClause = = NULL ) ;
2006-08-02 03:59:48 +02:00
foreach ( lc , selectStmt - > valuesLists )
{
List * sublist = ( List * ) lfirst ( lc ) ;
Improve handling of "UPDATE ... SET (column_list) = row_constructor".
Previously, the right-hand side of a multiple-column assignment, if it
wasn't a sub-SELECT, had to be a simple parenthesized expression list,
because gram.y was responsible for "bursting" the construct into
independent column assignments. This had the minor defect that you
couldn't write ROW (though you should be able to, since the standard says
this is a row constructor), and the rather larger defect that unlike other
uses of row constructors, we would not expand a "foo.*" item into multiple
columns.
Fix that by changing the RHS to be just "a_expr" in the grammar, leaving
it to transformMultiAssignRef to separate the elements of a RowExpr;
which it will do only after performing standard transformation of the
RowExpr, so that "foo.*" behaves as expected.
The key reason we didn't do that before was the hard-wired handling of
DEFAULT tokens (SetToDefault nodes). This patch deals with that issue by
allowing DEFAULT in any a_expr and having parse analysis throw an error
if SetToDefault is found in an unexpected place. That's an improvement
anyway since the error can be more specific than just "syntax error".
The SQL standard suggests that the RHS could be any a_expr yielding a
suitable row value. This patch doesn't really move the goal posts in that
respect --- you're still limited to RowExpr or a sub-SELECT --- but it does
fix the grammar restriction, so it provides some tangible progress towards
a full implementation. And the limitation is now documented by an explicit
error message rather than an unhelpful "syntax error".
Discussion: <8542.1479742008@sss.pgh.pa.us>
2016-11-22 21:19:57 +01:00
/*
* Do basic expression transformation ( same as a ROW ( ) expr , but
* allow SetToDefault at top level )
*/
sublist = transformExpressionList ( pstate , sublist ,
EXPR_KIND_VALUES , true ) ;
2006-08-02 03:59:48 +02:00
/*
* All the sublists must be the same length , * after *
* transformation ( which might expand ' * ' into multiple items ) .
* The VALUES RTE can ' t handle anything different .
*/
if ( sublist_length < 0 )
{
/* Remember post-transformation length of first sublist */
sublist_length = list_length ( sublist ) ;
}
else if ( sublist_length ! = list_length ( sublist ) )
{
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
2008-09-01 22:42:46 +02:00
errmsg ( " VALUES lists must all be the same length " ) ,
parser_errposition ( pstate ,
exprLocation ( ( Node * ) sublist ) ) ) ) ;
2006-08-02 03:59:48 +02:00
}
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
/*
* Prepare row for assignment to target table . We process any
* indirection on the target column specs normally but then strip
* off the resulting field / array assignment nodes , since we don ' t
* want the parsed statement to contain copies of those in each
* VALUES row . ( It ' s annoying to have to transform the
* indirection specs over and over like this , but avoiding it
* would take some really messy refactoring of
* transformAssignmentIndirection . )
*/
2006-08-02 03:59:48 +02:00
sublist = transformInsertRow ( pstate , sublist ,
stmt - > cols ,
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
icolumns , attrnos ,
true ) ;
2006-08-02 03:59:48 +02:00
2011-03-20 01:29:08 +01:00
/*
* We must assign collations now because assign_query_collations
* doesn ' t process rangetable entries . We just assign all the
* collations independently in each row , and don ' t worry about
2011-04-18 21:31:52 +02:00
* whether they are consistent vertically . The outer INSERT query
* isn ' t going to care about the collations of the VALUES columns ,
* so it ' s not worth the effort to identify a common collation for
* each one here . ( But note this does have one user - visible
* consequence : INSERT . . . VALUES won ' t complain about conflicting
* explicit COLLATEs in a column , whereas the same VALUES
* construct in another context would complain . )
2011-03-20 01:29:08 +01:00
*/
assign_list_collations ( pstate , sublist ) ;
2006-08-02 03:59:48 +02:00
exprsLists = lappend ( exprsLists , sublist ) ;
}
2011-04-18 21:31:52 +02:00
/*
Fix reporting of column typmods for multi-row VALUES constructs.
expandRTE() and get_rte_attribute_type() reported the exprType() and
exprTypmod() values of the expressions in the first row of the VALUES as
being the column type/typmod returned by the VALUES RTE. That's fine for
the data type, since we coerce all expressions in a column to have the same
common type. But we don't coerce them to have a common typmod, so it was
possible for rows after the first one to return values that violate the
claimed column typmod. This leads to the incorrect result seen in bug
#14448 from Hassan Mahmood, as well as some other corner-case misbehaviors.
The desired behavior is the same as we use in other type-unification
cases: report the common typmod if there is one, but otherwise return -1
indicating no particular constraint. It's cheap for transformValuesClause
to determine the common typmod while transforming a multi-row VALUES, but
it'd be less cheap for expandRTE() and get_rte_attribute_type() to
re-determine that info every time they're asked --- possibly a lot less
cheap, if the VALUES has many rows. Therefore, the best fix is to record
the common typmods explicitly in a list in the VALUES RTE, as we were
already doing for column collations. This looks quite a bit like what
we're doing for CTE RTEs, so we can save a little bit of space and code by
unifying the representation for those two RTE types. They both now share
coltypes/coltypmods/colcollations fields. (At some point it might seem
desirable to populate those fields for all RTE types; but right now it
looks like constructing them for other RTE types would add more code and
cycles than it would save.)
The RTE change requires a catversion bump, so this fix is only usable
in HEAD. If we fix this at all in the back branches, the patch will
need to look quite different.
Report: https://postgr.es/m/20161205143037.4377.60754@wrigleys.postgresql.org
Discussion: https://postgr.es/m/27429.1480968538@sss.pgh.pa.us
2016-12-08 17:40:02 +01:00
* Construct column type / typmod / collation lists for the VALUES RTE .
* Every expression in each column has been coerced to the type / typmod
* of the corresponding target column or subfield , so it ' s sufficient
* to look at the exprType / exprTypmod of the first row . We don ' t care
* about the collation labeling , so just fill in InvalidOid for that .
2011-04-18 21:31:52 +02:00
*/
Fix reporting of column typmods for multi-row VALUES constructs.
expandRTE() and get_rte_attribute_type() reported the exprType() and
exprTypmod() values of the expressions in the first row of the VALUES as
being the column type/typmod returned by the VALUES RTE. That's fine for
the data type, since we coerce all expressions in a column to have the same
common type. But we don't coerce them to have a common typmod, so it was
possible for rows after the first one to return values that violate the
claimed column typmod. This leads to the incorrect result seen in bug
#14448 from Hassan Mahmood, as well as some other corner-case misbehaviors.
The desired behavior is the same as we use in other type-unification
cases: report the common typmod if there is one, but otherwise return -1
indicating no particular constraint. It's cheap for transformValuesClause
to determine the common typmod while transforming a multi-row VALUES, but
it'd be less cheap for expandRTE() and get_rte_attribute_type() to
re-determine that info every time they're asked --- possibly a lot less
cheap, if the VALUES has many rows. Therefore, the best fix is to record
the common typmods explicitly in a list in the VALUES RTE, as we were
already doing for column collations. This looks quite a bit like what
we're doing for CTE RTEs, so we can save a little bit of space and code by
unifying the representation for those two RTE types. They both now share
coltypes/coltypmods/colcollations fields. (At some point it might seem
desirable to populate those fields for all RTE types; but right now it
looks like constructing them for other RTE types would add more code and
cycles than it would save.)
The RTE change requires a catversion bump, so this fix is only usable
in HEAD. If we fix this at all in the back branches, the patch will
need to look quite different.
Report: https://postgr.es/m/20161205143037.4377.60754@wrigleys.postgresql.org
Discussion: https://postgr.es/m/27429.1480968538@sss.pgh.pa.us
2016-12-08 17:40:02 +01:00
foreach ( lc , ( List * ) linitial ( exprsLists ) )
{
Node * val = ( Node * ) lfirst ( lc ) ;
coltypes = lappend_oid ( coltypes , exprType ( val ) ) ;
coltypmods = lappend_int ( coltypmods , exprTypmod ( val ) ) ;
colcollations = lappend_oid ( colcollations , InvalidOid ) ;
}
2011-04-18 21:31:52 +02:00
2006-08-02 03:59:48 +02:00
/*
2012-08-19 20:12:16 +02:00
* Ordinarily there can ' t be any current - level Vars in the expression
* lists , because the namespace was empty . . . but if we ' re inside
* CREATE RULE , then NEW / OLD references might appear . In that case we
* have to mark the VALUES RTE as LATERAL .
2006-08-02 03:59:48 +02:00
*/
2006-08-02 16:14:22 +02:00
if ( list_length ( pstate - > p_rtable ) ! = 1 & &
2006-08-02 15:58:52 +02:00
contain_vars_of_level ( ( Node * ) exprsLists , 0 ) )
2012-08-19 20:12:16 +02:00
lateral = true ;
2006-08-02 03:59:48 +02:00
/*
* Generate the VALUES RTE
*/
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
nsitem = addRangeTableEntryForValues ( pstate , exprsLists ,
coltypes , coltypmods , colcollations ,
NULL , lateral , true ) ;
addNSItemToQuery ( pstate , nsitem , true , false , false ) ;
2006-08-02 03:59:48 +02:00
/*
* Generate list of Vars referencing the RTE
*/
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
exprList = expandNSItemVars ( nsitem , 0 , - 1 , NULL ) ;
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
/*
* Re - apply any indirection on the target column specs to the Vars
*/
exprList = transformInsertRow ( pstate , exprList ,
stmt - > cols ,
icolumns , attrnos ,
false ) ;
2000-10-05 21:11:39 +02:00
}
2006-08-02 03:59:48 +02:00
else
{
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
/*
* Process INSERT . . . VALUES with a single VALUES sublist . We treat
* this case separately for efficiency . The sublist is just computed
* directly as the Query ' s targetlist , with no VALUES RTE . So it
* works just like a SELECT without any FROM .
2006-08-02 03:59:48 +02:00
*/
List * valuesLists = selectStmt - > valuesLists ;
1998-01-11 04:41:57 +01:00
2006-08-02 03:59:48 +02:00
Assert ( list_length ( valuesLists ) = = 1 ) ;
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
Assert ( selectStmt - > intoClause = = NULL ) ;
1999-07-19 02:26:20 +02:00
Improve handling of "UPDATE ... SET (column_list) = row_constructor".
Previously, the right-hand side of a multiple-column assignment, if it
wasn't a sub-SELECT, had to be a simple parenthesized expression list,
because gram.y was responsible for "bursting" the construct into
independent column assignments. This had the minor defect that you
couldn't write ROW (though you should be able to, since the standard says
this is a row constructor), and the rather larger defect that unlike other
uses of row constructors, we would not expand a "foo.*" item into multiple
columns.
Fix that by changing the RHS to be just "a_expr" in the grammar, leaving
it to transformMultiAssignRef to separate the elements of a RowExpr;
which it will do only after performing standard transformation of the
RowExpr, so that "foo.*" behaves as expected.
The key reason we didn't do that before was the hard-wired handling of
DEFAULT tokens (SetToDefault nodes). This patch deals with that issue by
allowing DEFAULT in any a_expr and having parse analysis throw an error
if SetToDefault is found in an unexpected place. That's an improvement
anyway since the error can be more specific than just "syntax error".
The SQL standard suggests that the RHS could be any a_expr yielding a
suitable row value. This patch doesn't really move the goal posts in that
respect --- you're still limited to RowExpr or a sub-SELECT --- but it does
fix the grammar restriction, so it provides some tangible progress towards
a full implementation. And the limitation is now documented by an explicit
error message rather than an unhelpful "syntax error".
Discussion: <8542.1479742008@sss.pgh.pa.us>
2016-11-22 21:19:57 +01:00
/*
* Do basic expression transformation ( same as a ROW ( ) expr , but allow
* SetToDefault at top level )
*/
2006-08-02 03:59:48 +02:00
exprList = transformExpressionList ( pstate ,
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
( List * ) linitial ( valuesLists ) ,
2017-01-16 21:23:11 +01:00
EXPR_KIND_VALUES_SINGLE ,
Improve handling of "UPDATE ... SET (column_list) = row_constructor".
Previously, the right-hand side of a multiple-column assignment, if it
wasn't a sub-SELECT, had to be a simple parenthesized expression list,
because gram.y was responsible for "bursting" the construct into
independent column assignments. This had the minor defect that you
couldn't write ROW (though you should be able to, since the standard says
this is a row constructor), and the rather larger defect that unlike other
uses of row constructors, we would not expand a "foo.*" item into multiple
columns.
Fix that by changing the RHS to be just "a_expr" in the grammar, leaving
it to transformMultiAssignRef to separate the elements of a RowExpr;
which it will do only after performing standard transformation of the
RowExpr, so that "foo.*" behaves as expected.
The key reason we didn't do that before was the hard-wired handling of
DEFAULT tokens (SetToDefault nodes). This patch deals with that issue by
allowing DEFAULT in any a_expr and having parse analysis throw an error
if SetToDefault is found in an unexpected place. That's an improvement
anyway since the error can be more specific than just "syntax error".
The SQL standard suggests that the RHS could be any a_expr yielding a
suitable row value. This patch doesn't really move the goal posts in that
respect --- you're still limited to RowExpr or a sub-SELECT --- but it does
fix the grammar restriction, so it provides some tangible progress towards
a full implementation. And the limitation is now documented by an explicit
error message rather than an unhelpful "syntax error".
Discussion: <8542.1479742008@sss.pgh.pa.us>
2016-11-22 21:19:57 +01:00
true ) ;
1999-07-19 02:26:20 +02:00
2006-08-02 03:59:48 +02:00
/* Prepare row for assignment to target table */
exprList = transformInsertRow ( pstate , exprList ,
stmt - > cols ,
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
icolumns , attrnos ,
false ) ;
2006-08-02 03:59:48 +02:00
}
1999-07-19 02:26:20 +02:00
2000-10-05 21:11:39 +02:00
/*
2006-08-02 03:59:48 +02:00
* Generate query ' s target list using the computed list of expressions .
2009-01-22 21:16:10 +01:00
* Also , mark all the target columns as needing insert permissions .
2000-10-05 21:11:39 +02:00
*/
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
rte = pstate - > p_target_nsitem - > p_rte ;
2006-08-02 03:59:48 +02:00
qry - > targetList = NIL ;
2019-02-28 20:25:01 +01:00
Assert ( list_length ( exprList ) < = list_length ( icolumns ) ) ;
forthree ( lc , exprList , icols , icolumns , attnos , attrnos )
1999-07-19 02:26:20 +02:00
{
2006-08-02 03:59:48 +02:00
Expr * expr = ( Expr * ) lfirst ( lc ) ;
2019-02-28 20:25:01 +01:00
ResTarget * col = lfirst_node ( ResTarget , icols ) ;
AttrNumber attr_num = ( AttrNumber ) lfirst_int ( attnos ) ;
2006-08-02 03:59:48 +02:00
TargetEntry * tle ;
2002-12-17 02:18:35 +01:00
2006-08-02 03:59:48 +02:00
tle = makeTargetEntry ( expr ,
2009-01-22 21:16:10 +01:00
attr_num ,
2006-08-02 03:59:48 +02:00
col - > name ,
false ) ;
qry - > targetList = lappend ( qry - > targetList , tle ) ;
2002-04-05 13:56:55 +02:00
2015-05-08 00:20:46 +02:00
rte - > insertedCols = bms_add_member ( rte - > insertedCols ,
2009-01-22 21:16:10 +01:00
attr_num - FirstLowInvalidHeapAttributeNumber ) ;
1999-07-19 02:26:20 +02:00
}
2006-08-12 04:52:06 +02:00
/*
Allow table-qualified variable names in ON CONFLICT ... WHERE.
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/16958-963f638020de271c@postgresql.org
2021-04-13 21:39:33 +02:00
* If we have any clauses yet to process , set the query namespace to
* contain only the target relation , removing any entries added in a
2006-08-12 04:52:06 +02:00
* sub - SELECT or VALUES list .
*/
Allow table-qualified variable names in ON CONFLICT ... WHERE.
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/16958-963f638020de271c@postgresql.org
2021-04-13 21:39:33 +02:00
if ( stmt - > onConflictClause | | stmt - > returningList )
2006-08-12 04:52:06 +02:00
{
2012-08-08 22:41:04 +02:00
pstate - > p_namespace = NIL ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
addNSItemToQuery ( pstate , pstate - > p_target_nsitem ,
false , true , true ) ;
Allow table-qualified variable names in ON CONFLICT ... WHERE.
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/16958-963f638020de271c@postgresql.org
2021-04-13 21:39:33 +02:00
}
/* Process ON CONFLICT, if any. */
if ( stmt - > onConflictClause )
qry - > onConflict = transformOnConflictClause ( pstate ,
stmt - > onConflictClause ) ;
/* Process RETURNING, if any. */
if ( stmt - > returningList )
2006-08-12 04:52:06 +02:00
qry - > returningList = transformReturningList ( pstate ,
stmt - > returningList ) ;
2000-10-05 21:11:39 +02:00
/* done building the range table and jointree */
qry - > rtable = pstate - > p_rtable ;
2007-06-24 00:12:52 +02:00
qry - > jointree = makeFromExpr ( pstate - > p_joinlist , NULL ) ;
2000-09-12 23:07:18 +02:00
2016-09-13 19:54:24 +02:00
qry - > hasTargetSRFs = pstate - > p_hasTargetSRFs ;
2007-06-24 00:12:52 +02:00
qry - > hasSubLinks = pstate - > p_hasSubLinks ;
2011-03-20 01:29:08 +01:00
assign_query_collations ( pstate , qry ) ;
2007-06-24 00:12:52 +02:00
return qry ;
}
/*
* Prepare an INSERT row for assignment to the target table .
*
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
* exprlist : transformed expressions for source values ; these might come from
* a VALUES row , or be Vars referencing a sub - SELECT or VALUES RTE output .
* stmtcols : original target - columns spec for INSERT ( we just test for NIL )
* icolumns : effective target - columns spec ( list of ResTarget )
* attrnos : integer column numbers ( must be same length as icolumns )
* strip_indirection : if true , remove any field / array assignment nodes
2007-06-24 00:12:52 +02:00
*/
2018-04-12 12:22:56 +02:00
static List *
2007-06-24 00:12:52 +02:00
transformInsertRow ( ParseState * pstate , List * exprlist ,
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
List * stmtcols , List * icolumns , List * attrnos ,
bool strip_indirection )
2007-06-24 00:12:52 +02:00
{
List * result ;
ListCell * lc ;
ListCell * icols ;
ListCell * attnos ;
2002-10-20 02:31:53 +02:00
1998-08-18 02:49:04 +02:00
/*
2007-06-24 00:12:52 +02:00
* Check length of expr list . It must not have more expressions than
* there are target columns . We allow fewer , but only if no explicit
* columns list was given ( the remaining columns are implicitly
* defaulted ) . Note we must check this * after * transformation because
* that could expand ' * ' into multiple items .
1998-08-18 02:49:04 +02:00
*/
2007-06-24 00:12:52 +02:00
if ( list_length ( exprlist ) > list_length ( icolumns ) )
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
2008-09-01 22:42:46 +02:00
errmsg ( " INSERT has more expressions than target columns " ) ,
parser_errposition ( pstate ,
exprLocation ( list_nth ( exprlist ,
list_length ( icolumns ) ) ) ) ) ) ;
2007-06-24 00:12:52 +02:00
if ( stmtcols ! = NIL & &
list_length ( exprlist ) < list_length ( icolumns ) )
2010-09-18 20:37:01 +02:00
{
/*
* We can get here for cases like INSERT . . . SELECT ( a , b , c ) FROM . . .
* where the user accidentally created a RowExpr instead of separate
* columns . Add a suitable hint if that seems to be the problem ,
* because the main error message is quite misleading for this case .
* ( If there ' s no stmtcols , you ' ll get something about data type
* mismatch , which is less misleading so we don ' t worry about giving a
* hint in that case . )
*/
2007-06-24 00:12:52 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
2008-09-01 22:42:46 +02:00
errmsg ( " INSERT has more target columns than expressions " ) ,
2010-09-18 20:37:01 +02:00
( ( list_length ( exprlist ) = = 1 & &
count_rowexpr_columns ( pstate , linitial ( exprlist ) ) = =
list_length ( icolumns ) ) ?
errhint ( " The insertion source is a row expression containing the same number of columns expected by the INSERT. Did you accidentally use extra parentheses? " ) : 0 ) ,
2008-09-01 22:42:46 +02:00
parser_errposition ( pstate ,
exprLocation ( list_nth ( icolumns ,
list_length ( exprlist ) ) ) ) ) ) ;
2010-09-18 20:37:01 +02:00
}
1998-08-18 02:49:04 +02:00
2007-06-24 00:12:52 +02:00
/*
* Prepare columns for assignment to target table .
*/
result = NIL ;
2019-02-28 20:25:01 +01:00
forthree ( lc , exprlist , icols , icolumns , attnos , attrnos )
1997-09-07 07:04:48 +02:00
{
2007-06-24 00:12:52 +02:00
Expr * expr = ( Expr * ) lfirst ( lc ) ;
2019-02-28 20:25:01 +01:00
ResTarget * col = lfirst_node ( ResTarget , icols ) ;
int attno = lfirst_int ( attnos ) ;
2000-09-12 23:07:18 +02:00
2007-06-24 00:12:52 +02:00
expr = transformAssignedExpr ( pstate , expr ,
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
EXPR_KIND_INSERT_TARGET ,
2007-06-24 00:12:52 +02:00
col - > name ,
2019-02-28 20:25:01 +01:00
attno ,
2007-06-24 00:12:52 +02:00
col - > indirection ,
col - > location ) ;
2000-09-12 23:07:18 +02:00
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
if ( strip_indirection )
{
while ( expr )
{
if ( IsA ( expr , FieldStore ) )
{
FieldStore * fstore = ( FieldStore * ) expr ;
expr = ( Expr * ) linitial ( fstore - > newvals ) ;
}
2019-02-01 16:50:32 +01:00
else if ( IsA ( expr , SubscriptingRef ) )
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
{
2019-02-01 16:50:32 +01:00
SubscriptingRef * sbsref = ( SubscriptingRef * ) expr ;
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
2019-02-01 16:50:32 +01:00
if ( sbsref - > refassgnexpr = = NULL )
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
break ;
2019-02-01 16:50:32 +01:00
expr = sbsref - > refassgnexpr ;
Make INSERT-from-multiple-VALUES-rows handle targetlist indirection better.
Previously, if an INSERT with multiple rows of VALUES had indirection
(array subscripting or field selection) in its target-columns list, the
parser handled that by applying transformAssignedExpr() to each element
of each VALUES row independently. This led to having ArrayRef assignment
nodes or FieldStore nodes in each row of the VALUES RTE. That works for
simple cases, but in bug #14265 Nuri Boardman points out that it fails
if there are multiple assignments to elements/fields of the same target
column. For such cases to work, rewriteTargetListIU() has to nest the
ArrayRefs or FieldStores together to produce a single expression to be
assigned to the column. But it failed to find them in the top-level
targetlist and issued an error about "multiple assignments to same column".
We could possibly fix this by teaching the rewriter to apply
rewriteTargetListIU to each VALUES row separately, but that would be messy
(it would change the output rowtype of the VALUES RTE, for example) and
inefficient. Instead, let's fix the parser so that the VALUES RTE outputs
are just the user-specified values, cast to the right type if necessary,
and then the ArrayRefs or FieldStores are applied in the top-level
targetlist to Vars representing the RTE's outputs. This is the same
parsetree representation already used for similar cases with INSERT/SELECT
syntax, so it allows simplifications in ruleutils.c, which no longer needs
to treat INSERT-from-multiple-VALUES as its own special case.
This implementation works by applying transformAssignedExpr to the VALUES
entries as before, and then stripping off any ArrayRefs or FieldStores it
adds. With lots of VALUES rows it would be noticeably more efficient to
not add those nodes in the first place. But that's just an optimization
not a bug fix, and there doesn't seem to be any good way to do it without
significant refactoring. (A non-invasive answer would be to apply
transformAssignedExpr + stripping to just the first VALUES row, and then
just forcibly cast remaining rows to the same data types exposed in the
first row. But this way would lead to different, not-INSERT-specific
errors being reported in casting failure cases, so it doesn't seem very
nice.) So leave that for later; this patch at least isn't making the
per-row parsing work worse, and it does make the finished parsetree
smaller, saving rewriter and planner work.
Catversion bump because stored rules containing such INSERTs would need
to change. Because of that, no back-patch, even though this is a very
long-standing bug.
Report: <20160727005725.7438.26021@wrigleys.postgresql.org>
Discussion: <9578.1469645245@sss.pgh.pa.us>
2016-08-03 22:37:03 +02:00
}
else
break ;
}
}
2007-06-24 00:12:52 +02:00
result = lappend ( result , expr ) ;
2000-09-12 23:07:18 +02:00
}
1996-07-09 08:22:35 +02:00
2007-06-24 00:12:52 +02:00
return result ;
1996-07-09 08:22:35 +02:00
}
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
/*
2016-09-22 23:34:21 +02:00
* transformOnConflictClause -
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
* transforms an OnConflictClause in an INSERT
*/
static OnConflictExpr *
transformOnConflictClause ( ParseState * pstate ,
OnConflictClause * onConflictClause )
{
Allow table-qualified variable names in ON CONFLICT ... WHERE.
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/16958-963f638020de271c@postgresql.org
2021-04-13 21:39:33 +02:00
ParseNamespaceItem * exclNSItem = NULL ;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
List * arbiterElems ;
Node * arbiterWhere ;
Oid arbiterConstraint ;
List * onConflictSet = NIL ;
Node * onConflictWhere = NULL ;
int exclRelIndex = 0 ;
List * exclRelTlist = NIL ;
OnConflictExpr * result ;
Allow table-qualified variable names in ON CONFLICT ... WHERE.
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/16958-963f638020de271c@postgresql.org
2021-04-13 21:39:33 +02:00
/*
* If this is ON CONFLICT . . . UPDATE , first create the range table entry
* for the EXCLUDED pseudo relation , so that that will be present while
* processing arbiter expressions . ( You can ' t actually reference it from
* there , but this provides a useful error message if you try . )
*/
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
if ( onConflictClause - > action = = ONCONFLICT_UPDATE )
{
Fix several bugs related to ON CONFLICT's EXCLUDED pseudo relation.
Four related issues:
1) attnos/varnos/resnos for EXCLUDED were out of sync when a column
after one dropped in the underlying relation was referenced.
2) References to whole-row variables (i.e. EXCLUDED.*) lead to errors.
3) It was possible to reference system columns in the EXCLUDED pseudo
relations, even though they would not have valid contents.
4) References to EXCLUDED were rewritten by the RLS machinery, as
EXCLUDED was treated as if it were the underlying relation.
To fix the first two issues, generate the excluded targetlist with
dropped columns in mind and add an entry for whole row
variables. Instead of unconditionally adding a wholerow entry we could
pull up the expression if needed, but doing it unconditionally seems
simpler. The wholerow entry is only really needed for ruleutils/EXPLAIN
support anyway.
The remaining two issues are addressed by changing the EXCLUDED RTE to
have relkind = composite. That fits with EXCLUDED not actually being a
real relation, and allows to treat it differently in the relevant
places. scanRTEForColumn now skips looking up system columns when the
RTE has a composite relkind; fireRIRrules() already had a corresponding
check, thereby preventing RLS expansion on EXCLUDED.
Also add tests for these issues, and improve a few comments around
excluded handling in setrefs.c.
Reported-By: Peter Geoghegan, Geoff Winkless
Author: Andres Freund, Amit Langote, Peter Geoghegan
Discussion: CAEzk6fdzJ3xYQZGbcuYM2rBd2BuDkUksmK=mY9UYYDugg_GgZg@mail.gmail.com,
CAM3SWZS+CauzbiCEcg-GdE6K6ycHE_Bz6Ksszy8AoixcMHOmsA@mail.gmail.com
Backpatch: 9.5, where ON CONFLICT was introduced
2015-10-03 15:12:10 +02:00
Relation targetrel = pstate - > p_target_relation ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
RangeTblEntry * exclRte ;
Fix several bugs related to ON CONFLICT's EXCLUDED pseudo relation.
Four related issues:
1) attnos/varnos/resnos for EXCLUDED were out of sync when a column
after one dropped in the underlying relation was referenced.
2) References to whole-row variables (i.e. EXCLUDED.*) lead to errors.
3) It was possible to reference system columns in the EXCLUDED pseudo
relations, even though they would not have valid contents.
4) References to EXCLUDED were rewritten by the RLS machinery, as
EXCLUDED was treated as if it were the underlying relation.
To fix the first two issues, generate the excluded targetlist with
dropped columns in mind and add an entry for whole row
variables. Instead of unconditionally adding a wholerow entry we could
pull up the expression if needed, but doing it unconditionally seems
simpler. The wholerow entry is only really needed for ruleutils/EXPLAIN
support anyway.
The remaining two issues are addressed by changing the EXCLUDED RTE to
have relkind = composite. That fits with EXCLUDED not actually being a
real relation, and allows to treat it differently in the relevant
places. scanRTEForColumn now skips looking up system columns when the
RTE has a composite relkind; fireRIRrules() already had a corresponding
check, thereby preventing RLS expansion on EXCLUDED.
Also add tests for these issues, and improve a few comments around
excluded handling in setrefs.c.
Reported-By: Peter Geoghegan, Geoff Winkless
Author: Andres Freund, Amit Langote, Peter Geoghegan
Discussion: CAEzk6fdzJ3xYQZGbcuYM2rBd2BuDkUksmK=mY9UYYDugg_GgZg@mail.gmail.com,
CAM3SWZS+CauzbiCEcg-GdE6K6ycHE_Bz6Ksszy8AoixcMHOmsA@mail.gmail.com
Backpatch: 9.5, where ON CONFLICT was introduced
2015-10-03 15:12:10 +02:00
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
exclNSItem = addRangeTableEntryForRelation ( pstate ,
targetrel ,
RowExclusiveLock ,
makeAlias ( " excluded " , NIL ) ,
false , false ) ;
exclRte = exclNSItem - > p_rte ;
exclRelIndex = exclNSItem - > p_rtindex ;
Allow table-qualified variable names in ON CONFLICT ... WHERE.
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/16958-963f638020de271c@postgresql.org
2021-04-13 21:39:33 +02:00
/*
* relkind is set to composite to signal that we ' re not dealing with
* an actual relation , and no permission checks are required on it .
* ( We ' ll check the actual target relation , instead . )
*/
Fix several bugs related to ON CONFLICT's EXCLUDED pseudo relation.
Four related issues:
1) attnos/varnos/resnos for EXCLUDED were out of sync when a column
after one dropped in the underlying relation was referenced.
2) References to whole-row variables (i.e. EXCLUDED.*) lead to errors.
3) It was possible to reference system columns in the EXCLUDED pseudo
relations, even though they would not have valid contents.
4) References to EXCLUDED were rewritten by the RLS machinery, as
EXCLUDED was treated as if it were the underlying relation.
To fix the first two issues, generate the excluded targetlist with
dropped columns in mind and add an entry for whole row
variables. Instead of unconditionally adding a wholerow entry we could
pull up the expression if needed, but doing it unconditionally seems
simpler. The wholerow entry is only really needed for ruleutils/EXPLAIN
support anyway.
The remaining two issues are addressed by changing the EXCLUDED RTE to
have relkind = composite. That fits with EXCLUDED not actually being a
real relation, and allows to treat it differently in the relevant
places. scanRTEForColumn now skips looking up system columns when the
RTE has a composite relkind; fireRIRrules() already had a corresponding
check, thereby preventing RLS expansion on EXCLUDED.
Also add tests for these issues, and improve a few comments around
excluded handling in setrefs.c.
Reported-By: Peter Geoghegan, Geoff Winkless
Author: Andres Freund, Amit Langote, Peter Geoghegan
Discussion: CAEzk6fdzJ3xYQZGbcuYM2rBd2BuDkUksmK=mY9UYYDugg_GgZg@mail.gmail.com,
CAM3SWZS+CauzbiCEcg-GdE6K6ycHE_Bz6Ksszy8AoixcMHOmsA@mail.gmail.com
Backpatch: 9.5, where ON CONFLICT was introduced
2015-10-03 15:12:10 +02:00
exclRte - > relkind = RELKIND_COMPOSITE_TYPE ;
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *.
When expanding an updatable view that is an INSERT's target, the rewriter
failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally
worked if the view was just "SELECT * FROM ...", as the transformation
would be a no-op in that case. With more complicated view targetlists,
this omission would often lead to "attribute ... has the wrong type" errors
or even crashes, as reported by Mario De Frutos Dieguez.
Fix by adding code to rewriteTargetView to fix up the data structure
correctly. The easiest way to update the exclRelTlist list is to rebuild
it from scratch looking at the new target relation, so factor the code
for that out of transformOnConflictClause to make it sharable.
In passing, avoid duplicate permissions checks against the EXCLUDED
pseudo-relation, and prevent useless view expansion of that relation's
dummy RTE. The latter is only known to happen (after this patch) in cases
where the query would fail later due to not having any INSTEAD OF triggers
for the view. But by exactly that token, it would create an unintended
and very poorly tested state of the query data structure, so it seems like
a good idea to prevent it from happening at all.
This has been broken since ON CONFLICT was introduced, so back-patch
to 9.5.
Dean Rasheed, based on an earlier patch by Amit Langote;
comment-kibitzing and back-patching by me
Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
exclRte - > requiredPerms = 0 ;
/* other permissions fields in exclRte are already empty */
Fix several bugs related to ON CONFLICT's EXCLUDED pseudo relation.
Four related issues:
1) attnos/varnos/resnos for EXCLUDED were out of sync when a column
after one dropped in the underlying relation was referenced.
2) References to whole-row variables (i.e. EXCLUDED.*) lead to errors.
3) It was possible to reference system columns in the EXCLUDED pseudo
relations, even though they would not have valid contents.
4) References to EXCLUDED were rewritten by the RLS machinery, as
EXCLUDED was treated as if it were the underlying relation.
To fix the first two issues, generate the excluded targetlist with
dropped columns in mind and add an entry for whole row
variables. Instead of unconditionally adding a wholerow entry we could
pull up the expression if needed, but doing it unconditionally seems
simpler. The wholerow entry is only really needed for ruleutils/EXPLAIN
support anyway.
The remaining two issues are addressed by changing the EXCLUDED RTE to
have relkind = composite. That fits with EXCLUDED not actually being a
real relation, and allows to treat it differently in the relevant
places. scanRTEForColumn now skips looking up system columns when the
RTE has a composite relkind; fireRIRrules() already had a corresponding
check, thereby preventing RLS expansion on EXCLUDED.
Also add tests for these issues, and improve a few comments around
excluded handling in setrefs.c.
Reported-By: Peter Geoghegan, Geoff Winkless
Author: Andres Freund, Amit Langote, Peter Geoghegan
Discussion: CAEzk6fdzJ3xYQZGbcuYM2rBd2BuDkUksmK=mY9UYYDugg_GgZg@mail.gmail.com,
CAM3SWZS+CauzbiCEcg-GdE6K6ycHE_Bz6Ksszy8AoixcMHOmsA@mail.gmail.com
Backpatch: 9.5, where ON CONFLICT was introduced
2015-10-03 15:12:10 +02:00
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *.
When expanding an updatable view that is an INSERT's target, the rewriter
failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally
worked if the view was just "SELECT * FROM ...", as the transformation
would be a no-op in that case. With more complicated view targetlists,
this omission would often lead to "attribute ... has the wrong type" errors
or even crashes, as reported by Mario De Frutos Dieguez.
Fix by adding code to rewriteTargetView to fix up the data structure
correctly. The easiest way to update the exclRelTlist list is to rebuild
it from scratch looking at the new target relation, so factor the code
for that out of transformOnConflictClause to make it sharable.
In passing, avoid duplicate permissions checks against the EXCLUDED
pseudo-relation, and prevent useless view expansion of that relation's
dummy RTE. The latter is only known to happen (after this patch) in cases
where the query would fail later due to not having any INSTEAD OF triggers
for the view. But by exactly that token, it would create an unintended
and very poorly tested state of the query data structure, so it seems like
a good idea to prevent it from happening at all.
This has been broken since ON CONFLICT was introduced, so back-patch
to 9.5.
Dean Rasheed, based on an earlier patch by Amit Langote;
comment-kibitzing and back-patching by me
Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
/* Create EXCLUDED rel's targetlist for use by EXPLAIN */
exclRelTlist = BuildOnConflictExcludedTargetlist ( targetrel ,
exclRelIndex ) ;
Allow table-qualified variable names in ON CONFLICT ... WHERE.
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/16958-963f638020de271c@postgresql.org
2021-04-13 21:39:33 +02:00
}
/* Process the arbiter clause, ON CONFLICT ON (...) */
transformOnConflictArbiter ( pstate , onConflictClause , & arbiterElems ,
& arbiterWhere , & arbiterConstraint ) ;
/* Process DO UPDATE */
if ( onConflictClause - > action = = ONCONFLICT_UPDATE )
{
/*
* Expressions in the UPDATE targetlist need to be handled like UPDATE
* not INSERT . We don ' t need to save / restore this because all INSERT
* expressions have been parsed already .
*/
pstate - > p_is_insert = false ;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
/*
Allow table-qualified variable names in ON CONFLICT ... WHERE.
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/16958-963f638020de271c@postgresql.org
2021-04-13 21:39:33 +02:00
* Add the EXCLUDED pseudo relation to the query namespace , making it
* available in the UPDATE subexpressions .
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
*/
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
addNSItemToQuery ( pstate , exclNSItem , false , true , true ) ;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *.
When expanding an updatable view that is an INSERT's target, the rewriter
failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally
worked if the view was just "SELECT * FROM ...", as the transformation
would be a no-op in that case. With more complicated view targetlists,
this omission would often lead to "attribute ... has the wrong type" errors
or even crashes, as reported by Mario De Frutos Dieguez.
Fix by adding code to rewriteTargetView to fix up the data structure
correctly. The easiest way to update the exclRelTlist list is to rebuild
it from scratch looking at the new target relation, so factor the code
for that out of transformOnConflictClause to make it sharable.
In passing, avoid duplicate permissions checks against the EXCLUDED
pseudo-relation, and prevent useless view expansion of that relation's
dummy RTE. The latter is only known to happen (after this patch) in cases
where the query would fail later due to not having any INSTEAD OF triggers
for the view. But by exactly that token, it would create an unintended
and very poorly tested state of the query data structure, so it seems like
a good idea to prevent it from happening at all.
This has been broken since ON CONFLICT was introduced, so back-patch
to 9.5.
Dean Rasheed, based on an earlier patch by Amit Langote;
comment-kibitzing and back-patching by me
Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
/*
* Now transform the UPDATE subexpressions .
*/
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
onConflictSet =
transformUpdateTargetList ( pstate , onConflictClause - > targetList ) ;
onConflictWhere = transformWhereClause ( pstate ,
onConflictClause - > whereClause ,
EXPR_KIND_WHERE , " WHERE " ) ;
Allow table-qualified variable names in ON CONFLICT ... WHERE.
Previously you could only use unqualified variable names here.
While that's not a functional deficiency, since only the target
table can be referenced, it's a surprising inconsistency with the
rules for partial-index predicates, on which this syntax is
supposedly modeled.
The fix for that is no harder than passing addToRelNameSpace = true
to addNSItemToQuery. However, it's really pretty bogus for
transformOnConflictArbiter and transformOnConflictClause to be
messing with the namespace item for the target table at all.
It's not theirs to manage, it results in duplicative creations of
namespace items, and transformOnConflictClause wasn't even doing
it quite correctly (that coding resulted in two nsitems for the
target table, since it hadn't cleaned out the existing one).
Hence, make transformInsertStmt responsible for setting up the
target nsitem once for both these clauses and RETURNING.
Also, arrange for ON CONFLICT ... UPDATE's "excluded" pseudo-relation
to be added to the rangetable before we run transformOnConflictArbiter.
This produces a more helpful HINT if someone writes "excluded.col"
in the arbiter expression.
Per bug #16958 from Lukas Eder. Although I agree this is a bug,
the consequences are hardly severe, so no back-patch.
Discussion: https://postgr.es/m/16958-963f638020de271c@postgresql.org
2021-04-13 21:39:33 +02:00
/*
* Remove the EXCLUDED pseudo relation from the query namespace , since
* it ' s not supposed to be available in RETURNING . ( Maybe someday we
* could allow that , and drop this step . )
*/
Assert ( ( ParseNamespaceItem * ) llast ( pstate - > p_namespace ) = = exclNSItem ) ;
pstate - > p_namespace = list_delete_last ( pstate - > p_namespace ) ;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
}
/* Finally, build ON CONFLICT DO [NOTHING | UPDATE] expression */
result = makeNode ( OnConflictExpr ) ;
result - > action = onConflictClause - > action ;
result - > arbiterElems = arbiterElems ;
result - > arbiterWhere = arbiterWhere ;
result - > constraint = arbiterConstraint ;
result - > onConflictSet = onConflictSet ;
result - > onConflictWhere = onConflictWhere ;
result - > exclRelIndex = exclRelIndex ;
result - > exclRelTlist = exclRelTlist ;
return result ;
}
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *.
When expanding an updatable view that is an INSERT's target, the rewriter
failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally
worked if the view was just "SELECT * FROM ...", as the transformation
would be a no-op in that case. With more complicated view targetlists,
this omission would often lead to "attribute ... has the wrong type" errors
or even crashes, as reported by Mario De Frutos Dieguez.
Fix by adding code to rewriteTargetView to fix up the data structure
correctly. The easiest way to update the exclRelTlist list is to rebuild
it from scratch looking at the new target relation, so factor the code
for that out of transformOnConflictClause to make it sharable.
In passing, avoid duplicate permissions checks against the EXCLUDED
pseudo-relation, and prevent useless view expansion of that relation's
dummy RTE. The latter is only known to happen (after this patch) in cases
where the query would fail later due to not having any INSTEAD OF triggers
for the view. But by exactly that token, it would create an unintended
and very poorly tested state of the query data structure, so it seems like
a good idea to prevent it from happening at all.
This has been broken since ON CONFLICT was introduced, so back-patch
to 9.5.
Dean Rasheed, based on an earlier patch by Amit Langote;
comment-kibitzing and back-patching by me
Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
/*
* BuildOnConflictExcludedTargetlist
* Create target list for the EXCLUDED pseudo - relation of ON CONFLICT ,
* representing the columns of targetrel with varno exclRelIndex .
*
* Note : Exported for use in the rewriter .
*/
List *
BuildOnConflictExcludedTargetlist ( Relation targetrel ,
Index exclRelIndex )
{
List * result = NIL ;
int attno ;
Var * var ;
TargetEntry * te ;
/*
* Note that resnos of the tlist must correspond to attnos of the
* underlying relation , hence we need entries for dropped columns too .
*/
for ( attno = 0 ; attno < RelationGetNumberOfAttributes ( targetrel ) ; attno + + )
{
Form_pg_attribute attr = TupleDescAttr ( targetrel - > rd_att , attno ) ;
char * name ;
if ( attr - > attisdropped )
{
/*
* can ' t use atttypid here , but it doesn ' t really matter what type
* the Const claims to be .
*/
var = ( Var * ) makeNullConst ( INT4OID , - 1 , InvalidOid ) ;
2018-09-18 21:08:28 +02:00
name = NULL ;
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *.
When expanding an updatable view that is an INSERT's target, the rewriter
failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally
worked if the view was just "SELECT * FROM ...", as the transformation
would be a no-op in that case. With more complicated view targetlists,
this omission would often lead to "attribute ... has the wrong type" errors
or even crashes, as reported by Mario De Frutos Dieguez.
Fix by adding code to rewriteTargetView to fix up the data structure
correctly. The easiest way to update the exclRelTlist list is to rebuild
it from scratch looking at the new target relation, so factor the code
for that out of transformOnConflictClause to make it sharable.
In passing, avoid duplicate permissions checks against the EXCLUDED
pseudo-relation, and prevent useless view expansion of that relation's
dummy RTE. The latter is only known to happen (after this patch) in cases
where the query would fail later due to not having any INSTEAD OF triggers
for the view. But by exactly that token, it would create an unintended
and very poorly tested state of the query data structure, so it seems like
a good idea to prevent it from happening at all.
This has been broken since ON CONFLICT was introduced, so back-patch
to 9.5.
Dean Rasheed, based on an earlier patch by Amit Langote;
comment-kibitzing and back-patching by me
Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
}
else
{
var = makeVar ( exclRelIndex , attno + 1 ,
attr - > atttypid , attr - > atttypmod ,
attr - > attcollation ,
0 ) ;
name = pstrdup ( NameStr ( attr - > attname ) ) ;
}
te = makeTargetEntry ( ( Expr * ) var ,
attno + 1 ,
name ,
false ) ;
result = lappend ( result , te ) ;
}
/*
* Add a whole - row - Var entry to support references to " EXCLUDED.* " . Like
* the other entries in the EXCLUDED tlist , its resno must match the Var ' s
* varattno , else the wrong things happen while resolving references in
* setrefs . c . This is against normal conventions for targetlists , but
* it ' s okay since we don ' t use this as a real tlist .
*/
var = makeVar ( exclRelIndex , InvalidAttrNumber ,
targetrel - > rd_rel - > reltype ,
- 1 , InvalidOid , 0 ) ;
te = makeTargetEntry ( ( Expr * ) var , InvalidAttrNumber , NULL , true ) ;
result = lappend ( result , te ) ;
return result ;
}
2010-09-18 20:37:01 +02:00
/*
* count_rowexpr_columns -
* get number of columns contained in a ROW ( ) expression ;
* return - 1 if expression isn ' t a RowExpr or a Var referencing one .
*
* This is currently used only for hint purposes , so we aren ' t terribly
* tense about recognizing all possible cases . The Var case is interesting
* because that ' s what we ' ll get in the INSERT . . . SELECT ( . . . ) case .
*/
static int
count_rowexpr_columns ( ParseState * pstate , Node * expr )
{
if ( expr = = NULL )
return - 1 ;
if ( IsA ( expr , RowExpr ) )
return list_length ( ( ( RowExpr * ) expr ) - > args ) ;
if ( IsA ( expr , Var ) )
{
Var * var = ( Var * ) expr ;
AttrNumber attnum = var - > varattno ;
if ( attnum > 0 & & var - > vartype = = RECORDOID )
{
RangeTblEntry * rte ;
rte = GetRTEByRangeTablePosn ( pstate , var - > varno , var - > varlevelsup ) ;
if ( rte - > rtekind = = RTE_SUBQUERY )
{
/* Subselect-in-FROM: examine sub-select's output expr */
TargetEntry * ste = get_tle_by_resno ( rte - > subquery - > targetList ,
attnum ) ;
if ( ste = = NULL | | ste - > resjunk )
return - 1 ;
expr = ( Node * ) ste - > expr ;
if ( IsA ( expr , RowExpr ) )
return list_length ( ( ( RowExpr * ) expr ) - > args ) ;
}
}
}
return - 1 ;
}
1996-07-09 08:22:35 +02:00
/*
* transformSelectStmt -
* transforms a Select Statement
2007-04-28 00:05:49 +02:00
*
2007-06-24 00:12:52 +02:00
* Note : this covers only cases with no set operations and no VALUES lists ;
* see below for the other cases .
1996-07-09 08:22:35 +02:00
*/
static Query *
1998-01-09 21:06:08 +01:00
transformSelectStmt ( ParseState * pstate , SelectStmt * stmt )
1996-07-09 08:22:35 +02:00
{
Query * qry = makeNode ( Query ) ;
2000-09-29 20:21:41 +02:00
Node * qual ;
2006-04-30 20:30:40 +02:00
ListCell * l ;
1997-09-07 07:04:48 +02:00
1996-07-09 08:22:35 +02:00
qry - > commandType = CMD_SELECT ;
2010-02-12 23:48:56 +01:00
/* process the WITH clause independently of all else */
2008-10-04 23:56:55 +02:00
if ( stmt - > withClause )
{
qry - > hasRecursive = stmt - > withClause - > recursive ;
qry - > cteList = transformWithClause ( pstate , stmt - > withClause ) ;
2011-02-26 00:56:23 +01:00
qry - > hasModifyingCTE = pstate - > p_hasModifyingCTE ;
2008-10-04 23:56:55 +02:00
}
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
/* Complain if we get called from someplace where INTO is not allowed */
if ( stmt - > intoClause )
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
errmsg ( " SELECT ... INTO is not allowed here " ) ,
parser_errposition ( pstate ,
exprLocation ( ( Node * ) stmt - > intoClause ) ) ) ) ;
2010-02-12 23:48:56 +01:00
/* make FOR UPDATE/FOR SHARE info available to addRangeTableEntry */
pstate - > p_locking_clause = stmt - > lockingClause ;
/* make WINDOW info available for window functions, too */
pstate - > p_windowdefs = stmt - > windowClause ;
2001-02-14 22:35:07 +01:00
/* process the FROM clause */
transformFromClause ( pstate , stmt - > fromClause ) ;
1996-07-09 08:22:35 +02:00
2001-11-05 06:00:14 +01:00
/* transform targetlist */
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
qry - > targetList = transformTargetList ( pstate , stmt - > targetList ,
EXPR_KIND_SELECT_TARGET ) ;
1996-07-09 08:22:35 +02:00
2003-05-06 02:20:33 +02:00
/* mark column origins */
markTargetListOrigins ( pstate , qry - > targetList ) ;
2001-11-05 06:00:14 +01:00
/* transform WHERE */
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
qual = transformWhereClause ( pstate , stmt - > whereClause ,
EXPR_KIND_WHERE , " WHERE " ) ;
1998-03-30 18:36:43 +02:00
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
/* initial processing of HAVING clause is much like WHERE clause */
2003-07-03 21:07:54 +02:00
qry - > havingQual = transformWhereClause ( pstate , stmt - > havingClause ,
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
EXPR_KIND_HAVING , " HAVING " ) ;
1998-03-30 18:36:43 +02:00
2003-06-15 18:42:08 +02:00
/*
* Transform sorting / grouping stuff . Do ORDER BY first because both
* transformGroupClause and transformDistinctClause need the results . Note
2008-08-01 00:47:56 +02:00
* that these functions can also change the targetList , so it ' s passed to
* them by reference .
2003-06-15 18:42:08 +02:00
*/
1997-09-01 07:56:34 +02:00
qry - > sortClause = transformSortClause ( pstate ,
stmt - > sortClause ,
2004-05-23 19:10:54 +02:00
& qry - > targetList ,
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
EXPR_KIND_ORDER_BY ,
2009-12-15 18:57:48 +01:00
false /* allow SQL92 rules */ ) ;
2000-01-27 19:11:50 +01:00
2003-06-15 18:42:08 +02:00
qry - > groupClause = transformGroupClause ( pstate ,
stmt - > groupClause ,
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
& qry - > groupingSets ,
2004-05-23 19:10:54 +02:00
& qry - > targetList ,
2008-12-28 19:54:01 +01:00
qry - > sortClause ,
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
EXPR_KIND_GROUP_BY ,
2009-12-15 18:57:48 +01:00
false /* allow SQL92 rules */ ) ;
Implement GROUP BY DISTINCT
With grouping sets, it's possible that some of the grouping sets are
duplicate. This is especially common with CUBE and ROLLUP clauses. For
example GROUP BY CUBE (a,b), CUBE (b,c) is equivalent to
GROUP BY GROUPING SETS (
(a, b, c),
(a, b, c),
(a, b, c),
(a, b),
(a, b),
(a, b),
(a),
(a),
(a),
(c, a),
(c, a),
(c, a),
(c),
(b, c),
(b),
()
)
Some of the grouping sets are calculated multiple times, which is mostly
unnecessary. This commit implements a new GROUP BY DISTINCT feature, as
defined in the SQL standard, which eliminates the duplicate sets.
Author: Vik Fearing
Reviewed-by: Erik Rijkers, Georgios Kokolatos, Tomas Vondra
Discussion: https://postgr.es/m/bf3805a8-d7d1-ae61-fece-761b7ff41ecc@postgresfriends.org
2021-03-18 17:45:38 +01:00
qry - > groupDistinct = stmt - > groupDistinct ;
2003-06-15 18:42:08 +02:00
2008-08-02 23:32:01 +02:00
if ( stmt - > distinctClause = = NIL )
{
qry - > distinctClause = NIL ;
qry - > hasDistinctOn = false ;
}
else if ( linitial ( stmt - > distinctClause ) = = NULL )
{
/* We had SELECT DISTINCT */
qry - > distinctClause = transformDistinctClause ( pstate ,
& qry - > targetList ,
2009-12-15 18:57:48 +01:00
qry - > sortClause ,
false ) ;
2008-08-02 23:32:01 +02:00
qry - > hasDistinctOn = false ;
}
else
{
/* We had SELECT DISTINCT ON */
qry - > distinctClause = transformDistinctOnClause ( pstate ,
stmt - > distinctClause ,
& qry - > targetList ,
qry - > sortClause ) ;
qry - > hasDistinctOn = true ;
}
1997-09-01 07:56:34 +02:00
2008-08-01 00:47:56 +02:00
/* transform LIMIT */
2003-07-03 21:07:54 +02:00
qry - > limitOffset = transformLimitClause ( pstate , stmt - > limitOffset ,
2020-04-07 22:22:13 +02:00
EXPR_KIND_OFFSET , " OFFSET " ,
stmt - > limitOption ) ;
2003-07-03 21:07:54 +02:00
qry - > limitCount = transformLimitClause ( pstate , stmt - > limitCount ,
2020-04-07 22:22:13 +02:00
EXPR_KIND_LIMIT , " LIMIT " ,
stmt - > limitOption ) ;
qry - > limitOption = stmt - > limitOption ;
2000-10-05 21:11:39 +02:00
2008-12-28 19:54:01 +01:00
/* transform window clauses after we have seen all window functions */
qry - > windowClause = transformWindowDefinitions ( pstate ,
pstate - > p_windowdefs ,
& qry - > targetList ) ;
Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column). This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on. For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison". Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.
To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command. In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.
Also, change creation of an unknown-type column in a relation from a
warning to a hard error. The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE. We want to forbid that because
it's nothing but a foot-gun.
This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have. Add a checking pass to detect that. While at it,
we can detect tables or composite types that would fail, essentially
for free. Those would fail safely anyway later on, but we might as
well fail earlier.
This patch is by me, but it owes something to previous investigations
by Rahila Syed. Also thanks to Ashutosh Bapat and Michael Paquier for
review.
Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 15:17:18 +01:00
/* resolve any still-unresolved output columns as being type text */
if ( pstate - > p_resolve_unknowns )
resolveTargetListUnknowns ( pstate , qry - > targetList ) ;
2003-01-17 04:25:04 +01:00
qry - > rtable = pstate - > p_rtable ;
qry - > jointree = makeFromExpr ( pstate - > p_joinlist , qual ) ;
1999-07-19 02:26:20 +02:00
qry - > hasSubLinks = pstate - > p_hasSubLinks ;
2008-12-28 19:54:01 +01:00
qry - > hasWindowFuncs = pstate - > p_hasWindowFuncs ;
2016-09-13 19:54:24 +02:00
qry - > hasTargetSRFs = pstate - > p_hasTargetSRFs ;
2012-02-08 19:15:02 +01:00
qry - > hasAggs = pstate - > p_hasAggs ;
2000-10-05 21:11:39 +02:00
2006-04-30 20:30:40 +02:00
foreach ( l , stmt - > lockingClause )
{
2009-10-28 15:55:47 +01:00
transformLockingClause ( pstate , qry ,
( LockingClause * ) lfirst ( l ) , false ) ;
2006-04-30 20:30:40 +02:00
}
2000-10-05 21:11:39 +02:00
2011-03-20 01:29:08 +01:00
assign_query_collations ( pstate , qry ) ;
2019-01-17 06:33:01 +01:00
/* this must be done after collations, for reliable comparison of exprs */
if ( pstate - > p_hasAggs | | qry - > groupClause | | qry - > groupingSets | | qry - > havingQual )
parseCheckAggregates ( pstate , qry ) ;
2000-10-05 21:11:39 +02:00
return qry ;
}
2006-08-02 03:59:48 +02:00
/*
* transformValuesClause -
* transforms a VALUES clause that ' s being used as a standalone SELECT
*
* We build a Query containing a VALUES RTE , rather as if one had written
2011-06-04 21:48:17 +02:00
* SELECT * FROM ( VALUES . . . ) AS " *VALUES* "
2006-08-02 03:59:48 +02:00
*/
static Query *
transformValuesClause ( ParseState * pstate , SelectStmt * stmt )
{
Query * qry = makeNode ( Query ) ;
2011-04-18 21:31:52 +02:00
List * exprsLists ;
Fix reporting of column typmods for multi-row VALUES constructs.
expandRTE() and get_rte_attribute_type() reported the exprType() and
exprTypmod() values of the expressions in the first row of the VALUES as
being the column type/typmod returned by the VALUES RTE. That's fine for
the data type, since we coerce all expressions in a column to have the same
common type. But we don't coerce them to have a common typmod, so it was
possible for rows after the first one to return values that violate the
claimed column typmod. This leads to the incorrect result seen in bug
#14448 from Hassan Mahmood, as well as some other corner-case misbehaviors.
The desired behavior is the same as we use in other type-unification
cases: report the common typmod if there is one, but otherwise return -1
indicating no particular constraint. It's cheap for transformValuesClause
to determine the common typmod while transforming a multi-row VALUES, but
it'd be less cheap for expandRTE() and get_rte_attribute_type() to
re-determine that info every time they're asked --- possibly a lot less
cheap, if the VALUES has many rows. Therefore, the best fix is to record
the common typmods explicitly in a list in the VALUES RTE, as we were
already doing for column collations. This looks quite a bit like what
we're doing for CTE RTEs, so we can save a little bit of space and code by
unifying the representation for those two RTE types. They both now share
coltypes/coltypmods/colcollations fields. (At some point it might seem
desirable to populate those fields for all RTE types; but right now it
looks like constructing them for other RTE types would add more code and
cycles than it would save.)
The RTE change requires a catversion bump, so this fix is only usable
in HEAD. If we fix this at all in the back branches, the patch will
need to look quite different.
Report: https://postgr.es/m/20161205143037.4377.60754@wrigleys.postgresql.org
Discussion: https://postgr.es/m/27429.1480968538@sss.pgh.pa.us
2016-12-08 17:40:02 +01:00
List * coltypes = NIL ;
List * coltypmods = NIL ;
List * colcollations = NIL ;
2008-08-29 01:09:48 +02:00
List * * colexprs = NULL ;
2006-08-02 03:59:48 +02:00
int sublist_length = - 1 ;
2012-08-19 20:12:16 +02:00
bool lateral = false ;
2019-12-26 17:16:42 +01:00
ParseNamespaceItem * nsitem ;
2006-08-02 03:59:48 +02:00
ListCell * lc ;
ListCell * lc2 ;
int i ;
qry - > commandType = CMD_SELECT ;
/* Most SELECT stuff doesn't apply in a VALUES clause */
Assert ( stmt - > distinctClause = = NIL ) ;
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
Assert ( stmt - > intoClause = = NULL ) ;
2006-08-02 03:59:48 +02:00
Assert ( stmt - > targetList = = NIL ) ;
Assert ( stmt - > fromClause = = NIL ) ;
Assert ( stmt - > whereClause = = NULL ) ;
Assert ( stmt - > groupClause = = NIL ) ;
Assert ( stmt - > havingClause = = NULL ) ;
2008-12-28 19:54:01 +01:00
Assert ( stmt - > windowClause = = NIL ) ;
2006-08-02 03:59:48 +02:00
Assert ( stmt - > op = = SETOP_NONE ) ;
2010-02-12 23:48:56 +01:00
/* process the WITH clause independently of all else */
2008-10-04 23:56:55 +02:00
if ( stmt - > withClause )
{
qry - > hasRecursive = stmt - > withClause - > recursive ;
qry - > cteList = transformWithClause ( pstate , stmt - > withClause ) ;
2011-02-26 00:56:23 +01:00
qry - > hasModifyingCTE = pstate - > p_hasModifyingCTE ;
2008-10-04 23:56:55 +02:00
}
2006-08-02 03:59:48 +02:00
/*
Improve handling of "UPDATE ... SET (column_list) = row_constructor".
Previously, the right-hand side of a multiple-column assignment, if it
wasn't a sub-SELECT, had to be a simple parenthesized expression list,
because gram.y was responsible for "bursting" the construct into
independent column assignments. This had the minor defect that you
couldn't write ROW (though you should be able to, since the standard says
this is a row constructor), and the rather larger defect that unlike other
uses of row constructors, we would not expand a "foo.*" item into multiple
columns.
Fix that by changing the RHS to be just "a_expr" in the grammar, leaving
it to transformMultiAssignRef to separate the elements of a RowExpr;
which it will do only after performing standard transformation of the
RowExpr, so that "foo.*" behaves as expected.
The key reason we didn't do that before was the hard-wired handling of
DEFAULT tokens (SetToDefault nodes). This patch deals with that issue by
allowing DEFAULT in any a_expr and having parse analysis throw an error
if SetToDefault is found in an unexpected place. That's an improvement
anyway since the error can be more specific than just "syntax error".
The SQL standard suggests that the RHS could be any a_expr yielding a
suitable row value. This patch doesn't really move the goal posts in that
respect --- you're still limited to RowExpr or a sub-SELECT --- but it does
fix the grammar restriction, so it provides some tangible progress towards
a full implementation. And the limitation is now documented by an explicit
error message rather than an unhelpful "syntax error".
Discussion: <8542.1479742008@sss.pgh.pa.us>
2016-11-22 21:19:57 +01:00
* For each row of VALUES , transform the raw expressions .
2011-04-18 21:31:52 +02:00
*
* Note that the intermediate representation we build is column - organized
* not row - organized . That simplifies the type and collation processing
* below .
2006-08-02 03:59:48 +02:00
*/
foreach ( lc , stmt - > valuesLists )
{
List * sublist = ( List * ) lfirst ( lc ) ;
Improve handling of "UPDATE ... SET (column_list) = row_constructor".
Previously, the right-hand side of a multiple-column assignment, if it
wasn't a sub-SELECT, had to be a simple parenthesized expression list,
because gram.y was responsible for "bursting" the construct into
independent column assignments. This had the minor defect that you
couldn't write ROW (though you should be able to, since the standard says
this is a row constructor), and the rather larger defect that unlike other
uses of row constructors, we would not expand a "foo.*" item into multiple
columns.
Fix that by changing the RHS to be just "a_expr" in the grammar, leaving
it to transformMultiAssignRef to separate the elements of a RowExpr;
which it will do only after performing standard transformation of the
RowExpr, so that "foo.*" behaves as expected.
The key reason we didn't do that before was the hard-wired handling of
DEFAULT tokens (SetToDefault nodes). This patch deals with that issue by
allowing DEFAULT in any a_expr and having parse analysis throw an error
if SetToDefault is found in an unexpected place. That's an improvement
anyway since the error can be more specific than just "syntax error".
The SQL standard suggests that the RHS could be any a_expr yielding a
suitable row value. This patch doesn't really move the goal posts in that
respect --- you're still limited to RowExpr or a sub-SELECT --- but it does
fix the grammar restriction, so it provides some tangible progress towards
a full implementation. And the limitation is now documented by an explicit
error message rather than an unhelpful "syntax error".
Discussion: <8542.1479742008@sss.pgh.pa.us>
2016-11-22 21:19:57 +01:00
/*
* Do basic expression transformation ( same as a ROW ( ) expr , but here
* we disallow SetToDefault )
*/
sublist = transformExpressionList ( pstate , sublist ,
EXPR_KIND_VALUES , false ) ;
2006-08-02 03:59:48 +02:00
/*
* All the sublists must be the same length , * after * transformation
* ( which might expand ' * ' into multiple items ) . The VALUES RTE can ' t
* handle anything different .
*/
if ( sublist_length < 0 )
{
/* Remember post-transformation length of first sublist */
sublist_length = list_length ( sublist ) ;
2011-04-18 21:31:52 +02:00
/* and allocate array for per-column lists */
2008-08-29 01:09:48 +02:00
colexprs = ( List * * ) palloc0 ( sublist_length * sizeof ( List * ) ) ;
2006-08-02 03:59:48 +02:00
}
else if ( sublist_length ! = list_length ( sublist ) )
{
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
2008-09-01 22:42:46 +02:00
errmsg ( " VALUES lists must all be the same length " ) ,
parser_errposition ( pstate ,
exprLocation ( ( Node * ) sublist ) ) ) ) ;
2006-08-02 03:59:48 +02:00
}
Improve handling of "UPDATE ... SET (column_list) = row_constructor".
Previously, the right-hand side of a multiple-column assignment, if it
wasn't a sub-SELECT, had to be a simple parenthesized expression list,
because gram.y was responsible for "bursting" the construct into
independent column assignments. This had the minor defect that you
couldn't write ROW (though you should be able to, since the standard says
this is a row constructor), and the rather larger defect that unlike other
uses of row constructors, we would not expand a "foo.*" item into multiple
columns.
Fix that by changing the RHS to be just "a_expr" in the grammar, leaving
it to transformMultiAssignRef to separate the elements of a RowExpr;
which it will do only after performing standard transformation of the
RowExpr, so that "foo.*" behaves as expected.
The key reason we didn't do that before was the hard-wired handling of
DEFAULT tokens (SetToDefault nodes). This patch deals with that issue by
allowing DEFAULT in any a_expr and having parse analysis throw an error
if SetToDefault is found in an unexpected place. That's an improvement
anyway since the error can be more specific than just "syntax error".
The SQL standard suggests that the RHS could be any a_expr yielding a
suitable row value. This patch doesn't really move the goal posts in that
respect --- you're still limited to RowExpr or a sub-SELECT --- but it does
fix the grammar restriction, so it provides some tangible progress towards
a full implementation. And the limitation is now documented by an explicit
error message rather than an unhelpful "syntax error".
Discussion: <8542.1479742008@sss.pgh.pa.us>
2016-11-22 21:19:57 +01:00
/* Build per-column expression lists */
2006-08-02 03:59:48 +02:00
i = 0 ;
foreach ( lc2 , sublist )
{
Node * col = ( Node * ) lfirst ( lc2 ) ;
2008-08-29 01:09:48 +02:00
colexprs [ i ] = lappend ( colexprs [ i ] , col ) ;
2006-08-02 03:59:48 +02:00
i + + ;
}
2011-04-18 21:31:52 +02:00
/* Release sub-list's cells to save memory */
list_free ( sublist ) ;
2006-08-02 03:59:48 +02:00
}
/*
* Now resolve the common types of the columns , and coerce everything to
Fix reporting of column typmods for multi-row VALUES constructs.
expandRTE() and get_rte_attribute_type() reported the exprType() and
exprTypmod() values of the expressions in the first row of the VALUES as
being the column type/typmod returned by the VALUES RTE. That's fine for
the data type, since we coerce all expressions in a column to have the same
common type. But we don't coerce them to have a common typmod, so it was
possible for rows after the first one to return values that violate the
claimed column typmod. This leads to the incorrect result seen in bug
#14448 from Hassan Mahmood, as well as some other corner-case misbehaviors.
The desired behavior is the same as we use in other type-unification
cases: report the common typmod if there is one, but otherwise return -1
indicating no particular constraint. It's cheap for transformValuesClause
to determine the common typmod while transforming a multi-row VALUES, but
it'd be less cheap for expandRTE() and get_rte_attribute_type() to
re-determine that info every time they're asked --- possibly a lot less
cheap, if the VALUES has many rows. Therefore, the best fix is to record
the common typmods explicitly in a list in the VALUES RTE, as we were
already doing for column collations. This looks quite a bit like what
we're doing for CTE RTEs, so we can save a little bit of space and code by
unifying the representation for those two RTE types. They both now share
coltypes/coltypmods/colcollations fields. (At some point it might seem
desirable to populate those fields for all RTE types; but right now it
looks like constructing them for other RTE types would add more code and
cycles than it would save.)
The RTE change requires a catversion bump, so this fix is only usable
in HEAD. If we fix this at all in the back branches, the patch will
need to look quite different.
Report: https://postgr.es/m/20161205143037.4377.60754@wrigleys.postgresql.org
Discussion: https://postgr.es/m/27429.1480968538@sss.pgh.pa.us
2016-12-08 17:40:02 +01:00
* those types . Then identify the common typmod and common collation , if
* any , of each column .
2011-04-18 21:31:52 +02:00
*
* We must do collation processing now because ( 1 ) assign_query_collations
* doesn ' t process rangetable entries , and ( 2 ) we need to label the VALUES
* RTE with column collations for use in the outer query . We don ' t
* consider conflict of implicit collations to be an error here ; instead
* the column will just show InvalidOid as its collation , and you ' ll get a
* failure later if that results in failure to resolve a collation .
*
* Note we modify the per - column expression lists in - place .
2006-08-02 03:59:48 +02:00
*/
for ( i = 0 ; i < sublist_length ; i + + )
{
2011-04-18 21:31:52 +02:00
Oid coltype ;
2020-10-27 17:39:23 +01:00
int32 coltypmod ;
2011-04-18 21:31:52 +02:00
Oid colcoll ;
2006-08-02 03:59:48 +02:00
2011-04-18 21:31:52 +02:00
coltype = select_common_type ( pstate , colexprs [ i ] , " VALUES " , NULL ) ;
2006-08-02 03:59:48 +02:00
2011-04-18 21:31:52 +02:00
foreach ( lc , colexprs [ i ] )
2006-08-02 03:59:48 +02:00
{
2011-04-18 21:31:52 +02:00
Node * col = ( Node * ) lfirst ( lc ) ;
2006-08-02 03:59:48 +02:00
2011-04-18 21:31:52 +02:00
col = coerce_to_common_type ( pstate , col , coltype , " VALUES " ) ;
lfirst ( lc ) = ( void * ) col ;
2006-08-02 03:59:48 +02:00
}
2020-10-27 17:39:23 +01:00
coltypmod = select_common_typmod ( pstate , colexprs [ i ] , coltype ) ;
2011-04-18 21:31:52 +02:00
colcoll = select_common_collation ( pstate , colexprs [ i ] , true ) ;
Fix reporting of column typmods for multi-row VALUES constructs.
expandRTE() and get_rte_attribute_type() reported the exprType() and
exprTypmod() values of the expressions in the first row of the VALUES as
being the column type/typmod returned by the VALUES RTE. That's fine for
the data type, since we coerce all expressions in a column to have the same
common type. But we don't coerce them to have a common typmod, so it was
possible for rows after the first one to return values that violate the
claimed column typmod. This leads to the incorrect result seen in bug
#14448 from Hassan Mahmood, as well as some other corner-case misbehaviors.
The desired behavior is the same as we use in other type-unification
cases: report the common typmod if there is one, but otherwise return -1
indicating no particular constraint. It's cheap for transformValuesClause
to determine the common typmod while transforming a multi-row VALUES, but
it'd be less cheap for expandRTE() and get_rte_attribute_type() to
re-determine that info every time they're asked --- possibly a lot less
cheap, if the VALUES has many rows. Therefore, the best fix is to record
the common typmods explicitly in a list in the VALUES RTE, as we were
already doing for column collations. This looks quite a bit like what
we're doing for CTE RTEs, so we can save a little bit of space and code by
unifying the representation for those two RTE types. They both now share
coltypes/coltypmods/colcollations fields. (At some point it might seem
desirable to populate those fields for all RTE types; but right now it
looks like constructing them for other RTE types would add more code and
cycles than it would save.)
The RTE change requires a catversion bump, so this fix is only usable
in HEAD. If we fix this at all in the back branches, the patch will
need to look quite different.
Report: https://postgr.es/m/20161205143037.4377.60754@wrigleys.postgresql.org
Discussion: https://postgr.es/m/27429.1480968538@sss.pgh.pa.us
2016-12-08 17:40:02 +01:00
coltypes = lappend_oid ( coltypes , coltype ) ;
coltypmods = lappend_int ( coltypmods , coltypmod ) ;
colcollations = lappend_oid ( colcollations , colcoll ) ;
2011-04-18 21:31:52 +02:00
}
2011-03-20 01:29:08 +01:00
2011-04-18 21:31:52 +02:00
/*
* Finally , rearrange the coerced expressions into row - organized lists .
*/
exprsLists = NIL ;
foreach ( lc , colexprs [ 0 ] )
{
Node * col = ( Node * ) lfirst ( lc ) ;
List * sublist ;
sublist = list_make1 ( col ) ;
exprsLists = lappend ( exprsLists , sublist ) ;
}
list_free ( colexprs [ 0 ] ) ;
for ( i = 1 ; i < sublist_length ; i + + )
{
forboth ( lc , colexprs [ i ] , lc2 , exprsLists )
{
Node * col = ( Node * ) lfirst ( lc ) ;
List * sublist = lfirst ( lc2 ) ;
2020-10-17 08:38:39 +02:00
sublist = lappend ( sublist , col ) ;
2011-04-18 21:31:52 +02:00
}
list_free ( colexprs [ i ] ) ;
2006-08-02 03:59:48 +02:00
}
2012-08-19 20:12:16 +02:00
/*
* Ordinarily there can ' t be any current - level Vars in the expression
* lists , because the namespace was empty . . . but if we ' re inside CREATE
* RULE , then NEW / OLD references might appear . In that case we have to
* mark the VALUES RTE as LATERAL .
*/
if ( pstate - > p_rtable ! = NIL & &
contain_vars_of_level ( ( Node * ) exprsLists , 0 ) )
lateral = true ;
2006-08-02 03:59:48 +02:00
/*
* Generate the VALUES RTE
*/
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
nsitem = addRangeTableEntryForValues ( pstate , exprsLists ,
coltypes , coltypmods , colcollations ,
NULL , lateral , true ) ;
addNSItemToQuery ( pstate , nsitem , true , true , true ) ;
2006-08-02 03:59:48 +02:00
/*
* Generate a targetlist as though expanding " * "
*/
Assert ( pstate - > p_next_resno = = 1 ) ;
2019-12-26 17:16:42 +01:00
qry - > targetList = expandNSItemAttrs ( pstate , nsitem , 0 , - 1 ) ;
2006-08-02 03:59:48 +02:00
/*
2006-09-18 02:52:14 +02:00
* The grammar allows attaching ORDER BY , LIMIT , and FOR UPDATE to a
2006-08-02 03:59:48 +02:00
* VALUES , so cope .
*/
qry - > sortClause = transformSortClause ( pstate ,
stmt - > sortClause ,
& qry - > targetList ,
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
EXPR_KIND_ORDER_BY ,
2009-12-15 18:57:48 +01:00
false /* allow SQL92 rules */ ) ;
2006-08-02 03:59:48 +02:00
qry - > limitOffset = transformLimitClause ( pstate , stmt - > limitOffset ,
2020-04-07 22:22:13 +02:00
EXPR_KIND_OFFSET , " OFFSET " ,
stmt - > limitOption ) ;
2006-08-02 03:59:48 +02:00
qry - > limitCount = transformLimitClause ( pstate , stmt - > limitCount ,
2020-04-07 22:22:13 +02:00
EXPR_KIND_LIMIT , " LIMIT " ,
stmt - > limitOption ) ;
qry - > limitOption = stmt - > limitOption ;
2006-08-02 03:59:48 +02:00
if ( stmt - > lockingClause )
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s cannot be applied to VALUES " ,
LCS_asString ( ( ( LockingClause * )
linitial ( stmt - > lockingClause ) ) - > strength ) ) ) ) ;
2006-08-02 03:59:48 +02:00
qry - > rtable = pstate - > p_rtable ;
qry - > jointree = makeFromExpr ( pstate - > p_joinlist , NULL ) ;
qry - > hasSubLinks = pstate - > p_hasSubLinks ;
2011-03-20 01:29:08 +01:00
assign_query_collations ( pstate , qry ) ;
2006-08-02 03:59:48 +02:00
return qry ;
}
2000-10-05 21:11:39 +02:00
/*
2007-06-24 00:12:52 +02:00
* transformSetOperationStmt -
2000-11-05 01:15:54 +01:00
* transforms a set - operations tree
2000-10-05 21:11:39 +02:00
*
2000-11-05 01:15:54 +01:00
* A set - operation tree is just a SELECT , but with UNION / INTERSECT / EXCEPT
2000-10-05 21:11:39 +02:00
* structure to it . We must transform each leaf SELECT and build up a top -
* level Query that contains the leaf SELECTs as subqueries in its rangetable .
2000-11-05 01:15:54 +01:00
* The tree of set operations is converted into the setOperations field of
* the top - level Query .
2000-10-05 21:11:39 +02:00
*/
2001-05-18 23:24:20 +02:00
static Query *
transformSetOperationStmt ( ParseState * pstate , SelectStmt * stmt )
2000-10-05 21:11:39 +02:00
{
Query * qry = makeNode ( Query ) ;
SelectStmt * leftmostSelect ;
2000-11-05 02:42:07 +01:00
int leftmostRTI ;
2000-10-05 21:11:39 +02:00
Query * leftmostQuery ;
2000-11-05 01:15:54 +01:00
SetOperationStmt * sostmt ;
2000-10-05 21:11:39 +02:00
List * sortClause ;
Node * limitOffset ;
Node * limitCount ;
2006-04-30 20:30:40 +02:00
List * lockingClause ;
2012-07-31 23:56:21 +02:00
WithClause * withClause ;
2000-11-05 01:15:54 +01:00
Node * node ;
2004-05-26 06:41:50 +02:00
ListCell * left_tlist ,
2006-08-10 04:36:29 +02:00
* lct ,
* lcm ,
2011-02-08 22:04:18 +01:00
* lcc ,
2006-04-30 20:30:40 +02:00
* l ;
2004-05-26 06:41:50 +02:00
List * targetvars ,
2001-02-15 02:10:28 +01:00
* targetnames ,
2012-08-08 22:41:04 +02:00
* sv_namespace ;
2009-01-22 21:16:10 +01:00
int sv_rtable_length ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
ParseNamespaceItem * jnsitem ;
ParseNamespaceColumn * sortnscolumns ;
int sortcolindex ;
2000-10-05 21:11:39 +02:00
int tllen ;
qry - > commandType = CMD_SELECT ;
1998-05-29 15:39:30 +02:00
/*
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
* Find leftmost leaf SelectStmt . We currently only need to do this in
* order to deliver a suitable error message if there ' s an INTO clause
* there , implying the set - op tree is in a context that doesn ' t allow
* INTO . ( transformSetOperationTree would throw error anyway , but it
* seems worth the trouble to throw a different error for non - leftmost
* INTO , so we produce that error in transformSetOperationTree . )
1998-05-29 15:39:30 +02:00
*/
2000-11-05 01:15:54 +01:00
leftmostSelect = stmt - > larg ;
while ( leftmostSelect & & leftmostSelect - > op ! = SETOP_NONE )
leftmostSelect = leftmostSelect - > larg ;
Assert ( leftmostSelect & & IsA ( leftmostSelect , SelectStmt ) & &
leftmostSelect - > larg = = NULL ) ;
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
if ( leftmostSelect - > intoClause )
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
errmsg ( " SELECT ... INTO is not allowed here " ) ,
parser_errposition ( pstate ,
exprLocation ( ( Node * ) leftmostSelect - > intoClause ) ) ) ) ;
2000-10-05 21:11:39 +02:00
2000-11-05 01:15:54 +01:00
/*
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
* We need to extract ORDER BY and other top - level clauses here and not
2000-11-05 01:15:54 +01:00
* let transformSetOperationTree ( ) see them - - - else it ' ll just recurse
* right back here !
*/
sortClause = stmt - > sortClause ;
limitOffset = stmt - > limitOffset ;
limitCount = stmt - > limitCount ;
2005-08-01 22:31:16 +02:00
lockingClause = stmt - > lockingClause ;
2012-07-31 23:56:21 +02:00
withClause = stmt - > withClause ;
2000-11-05 01:15:54 +01:00
stmt - > sortClause = NIL ;
stmt - > limitOffset = NULL ;
stmt - > limitCount = NULL ;
2006-04-30 20:30:40 +02:00
stmt - > lockingClause = NIL ;
2012-07-31 23:56:21 +02:00
stmt - > withClause = NULL ;
2000-11-05 01:15:54 +01:00
2005-04-28 23:47:18 +02:00
/* We don't support FOR UPDATE/SHARE with set ops at the moment. */
2005-08-01 22:31:16 +02:00
if ( lockingClause )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s is not allowed with UNION/INTERSECT/EXCEPT " ,
LCS_asString ( ( ( LockingClause * )
linitial ( lockingClause ) ) - > strength ) ) ) ) ;
Hi!
INTERSECT and EXCEPT is available for postgresql-v6.4!
The patch against v6.4 is included at the end of the current text
(in uuencoded form!)
I also included the text of my Master's Thesis. (a postscript
version). I hope that you find something of it useful and would be
happy if parts of it find their way into the PostgreSQL documentation
project (If so, tell me, then I send the sources of the document!)
The contents of the document are:
-) The first chapter might be of less interest as it gives only an
overview on SQL.
-) The second chapter gives a description on much of PostgreSQL's
features (like user defined types etc. and how to use these features)
-) The third chapter starts with an overview of PostgreSQL's internal
structure with focus on the stages a query has to pass (i.e. parser,
planner/optimizer, executor). Then a detailed description of the
implementation of the Having clause and the Intersect/Except logic is
given.
Originally I worked on v6.3.2 but never found time enough to prepare
and post a patch. Now I applied the changes to v6.4 to get Intersect
and Except working with the new version. Chapter 3 of my documentation
deals with the changes against v6.3.2, so keep that in mind when
comparing the parts of the code printed there with the patched sources
of v6.4.
Here are some remarks on the patch. There are some things that have
still to be done but at the moment I don't have time to do them
myself. (I'm doing my military service at the moment) Sorry for that
:-(
-) I used a rewrite technique for the implementation of the Except/Intersect
logic which rewrites the query to a semantically equivalent query before
it is handed to the rewrite system (for views, rules etc.), planner,
executor etc.
-) In v6.3.2 the types of the attributes of two select statements
connected by the UNION keyword had to match 100%. In v6.4 the types
only need to be familiar (i.e. int and float can be mixed). Since this
feature did not exist when I worked on Intersect/Except it
does not work correctly for Except/Intersect queries WHEN USED IN
COMBINATION WITH UNIONS! (i.e. sometimes the wrong type is used for the
resulting table. This is because until now the types of the attributes of
the first select statement have been used for the resulting table.
When Intersects and/or Excepts are used in combination with Unions it
might happen, that the first select statement of the original query
appears at another position in the query which will be executed. The reason
for this is the technique used for the implementation of
Except/Intersect which does a query rewrite!)
NOTE: It is NOT broken for pure UNION queries and pure INTERSECT/EXCEPT
queries!!!
-) I had to add the field intersect_clause to some data structures
but did not find time to implement printfuncs for the new field.
This does NOT break the debug modes but when an Except/Intersect
is used the query debug output will be the already rewritten query.
-) Massive changes to the grammar rules for SELECT and INSERT statements
have been necessary (see comments in gram.y and documentation for
deatails) in order to be able to use mixed queries like
(SELECT ... UNION (SELECT ... EXCEPT SELECT)) INTERSECT SELECT...;
-) When using UNION/EXCEPT/INTERSECT you will get:
NOTICE: equal: "Don't know if nodes of type xxx are equal".
I did not have time to add comparsion support for all the needed nodes,
but the default behaviour of the function equal met my requirements.
I did not dare to supress this message!
That's the reason why the regression test for union will fail: These
messages are also included in the union.out file!
-) Somebody of you changed the union_planner() function for v6.4
(I copied the targetlist to new_tlist and that was removed and
replaced by a cleanup of the original targetlist). These chnages
violated some having queries executed against views so I changed
it back again. I did not have time to examine the differences between the
two versions but now it works :-)
If you want to find out, try the file queries/view_having.sql on
both versions and compare the results . Two queries won't produce a
correct result with your version.
regards
Stefan
1999-01-18 01:10:17 +01:00
2012-07-31 23:56:21 +02:00
/* Process the WITH clause independently of all else */
if ( withClause )
{
qry - > hasRecursive = withClause - > recursive ;
qry - > cteList = transformWithClause ( pstate , withClause ) ;
qry - > hasModifyingCTE = pstate - > p_hasModifyingCTE ;
}
Hi!
INTERSECT and EXCEPT is available for postgresql-v6.4!
The patch against v6.4 is included at the end of the current text
(in uuencoded form!)
I also included the text of my Master's Thesis. (a postscript
version). I hope that you find something of it useful and would be
happy if parts of it find their way into the PostgreSQL documentation
project (If so, tell me, then I send the sources of the document!)
The contents of the document are:
-) The first chapter might be of less interest as it gives only an
overview on SQL.
-) The second chapter gives a description on much of PostgreSQL's
features (like user defined types etc. and how to use these features)
-) The third chapter starts with an overview of PostgreSQL's internal
structure with focus on the stages a query has to pass (i.e. parser,
planner/optimizer, executor). Then a detailed description of the
implementation of the Having clause and the Intersect/Except logic is
given.
Originally I worked on v6.3.2 but never found time enough to prepare
and post a patch. Now I applied the changes to v6.4 to get Intersect
and Except working with the new version. Chapter 3 of my documentation
deals with the changes against v6.3.2, so keep that in mind when
comparing the parts of the code printed there with the patched sources
of v6.4.
Here are some remarks on the patch. There are some things that have
still to be done but at the moment I don't have time to do them
myself. (I'm doing my military service at the moment) Sorry for that
:-(
-) I used a rewrite technique for the implementation of the Except/Intersect
logic which rewrites the query to a semantically equivalent query before
it is handed to the rewrite system (for views, rules etc.), planner,
executor etc.
-) In v6.3.2 the types of the attributes of two select statements
connected by the UNION keyword had to match 100%. In v6.4 the types
only need to be familiar (i.e. int and float can be mixed). Since this
feature did not exist when I worked on Intersect/Except it
does not work correctly for Except/Intersect queries WHEN USED IN
COMBINATION WITH UNIONS! (i.e. sometimes the wrong type is used for the
resulting table. This is because until now the types of the attributes of
the first select statement have been used for the resulting table.
When Intersects and/or Excepts are used in combination with Unions it
might happen, that the first select statement of the original query
appears at another position in the query which will be executed. The reason
for this is the technique used for the implementation of
Except/Intersect which does a query rewrite!)
NOTE: It is NOT broken for pure UNION queries and pure INTERSECT/EXCEPT
queries!!!
-) I had to add the field intersect_clause to some data structures
but did not find time to implement printfuncs for the new field.
This does NOT break the debug modes but when an Except/Intersect
is used the query debug output will be the already rewritten query.
-) Massive changes to the grammar rules for SELECT and INSERT statements
have been necessary (see comments in gram.y and documentation for
deatails) in order to be able to use mixed queries like
(SELECT ... UNION (SELECT ... EXCEPT SELECT)) INTERSECT SELECT...;
-) When using UNION/EXCEPT/INTERSECT you will get:
NOTICE: equal: "Don't know if nodes of type xxx are equal".
I did not have time to add comparsion support for all the needed nodes,
but the default behaviour of the function equal met my requirements.
I did not dare to supress this message!
That's the reason why the regression test for union will fail: These
messages are also included in the union.out file!
-) Somebody of you changed the union_planner() function for v6.4
(I copied the targetlist to new_tlist and that was removed and
replaced by a cleanup of the original targetlist). These chnages
violated some having queries executed against views so I changed
it back again. I did not have time to examine the differences between the
two versions but now it works :-)
If you want to find out, try the file queries/view_having.sql on
both versions and compare the results . Two queries won't produce a
correct result with your version.
regards
Stefan
1999-01-18 01:10:17 +01:00
/*
2000-10-05 21:11:39 +02:00
* Recursively transform the components of the tree .
Hi!
INTERSECT and EXCEPT is available for postgresql-v6.4!
The patch against v6.4 is included at the end of the current text
(in uuencoded form!)
I also included the text of my Master's Thesis. (a postscript
version). I hope that you find something of it useful and would be
happy if parts of it find their way into the PostgreSQL documentation
project (If so, tell me, then I send the sources of the document!)
The contents of the document are:
-) The first chapter might be of less interest as it gives only an
overview on SQL.
-) The second chapter gives a description on much of PostgreSQL's
features (like user defined types etc. and how to use these features)
-) The third chapter starts with an overview of PostgreSQL's internal
structure with focus on the stages a query has to pass (i.e. parser,
planner/optimizer, executor). Then a detailed description of the
implementation of the Having clause and the Intersect/Except logic is
given.
Originally I worked on v6.3.2 but never found time enough to prepare
and post a patch. Now I applied the changes to v6.4 to get Intersect
and Except working with the new version. Chapter 3 of my documentation
deals with the changes against v6.3.2, so keep that in mind when
comparing the parts of the code printed there with the patched sources
of v6.4.
Here are some remarks on the patch. There are some things that have
still to be done but at the moment I don't have time to do them
myself. (I'm doing my military service at the moment) Sorry for that
:-(
-) I used a rewrite technique for the implementation of the Except/Intersect
logic which rewrites the query to a semantically equivalent query before
it is handed to the rewrite system (for views, rules etc.), planner,
executor etc.
-) In v6.3.2 the types of the attributes of two select statements
connected by the UNION keyword had to match 100%. In v6.4 the types
only need to be familiar (i.e. int and float can be mixed). Since this
feature did not exist when I worked on Intersect/Except it
does not work correctly for Except/Intersect queries WHEN USED IN
COMBINATION WITH UNIONS! (i.e. sometimes the wrong type is used for the
resulting table. This is because until now the types of the attributes of
the first select statement have been used for the resulting table.
When Intersects and/or Excepts are used in combination with Unions it
might happen, that the first select statement of the original query
appears at another position in the query which will be executed. The reason
for this is the technique used for the implementation of
Except/Intersect which does a query rewrite!)
NOTE: It is NOT broken for pure UNION queries and pure INTERSECT/EXCEPT
queries!!!
-) I had to add the field intersect_clause to some data structures
but did not find time to implement printfuncs for the new field.
This does NOT break the debug modes but when an Except/Intersect
is used the query debug output will be the already rewritten query.
-) Massive changes to the grammar rules for SELECT and INSERT statements
have been necessary (see comments in gram.y and documentation for
deatails) in order to be able to use mixed queries like
(SELECT ... UNION (SELECT ... EXCEPT SELECT)) INTERSECT SELECT...;
-) When using UNION/EXCEPT/INTERSECT you will get:
NOTICE: equal: "Don't know if nodes of type xxx are equal".
I did not have time to add comparsion support for all the needed nodes,
but the default behaviour of the function equal met my requirements.
I did not dare to supress this message!
That's the reason why the regression test for union will fail: These
messages are also included in the union.out file!
-) Somebody of you changed the union_planner() function for v6.4
(I copied the targetlist to new_tlist and that was removed and
replaced by a cleanup of the original targetlist). These chnages
violated some having queries executed against views so I changed
it back again. I did not have time to examine the differences between the
two versions but now it works :-)
If you want to find out, try the file queries/view_having.sql on
both versions and compare the results . Two queries won't produce a
correct result with your version.
regards
Stefan
1999-01-18 01:10:17 +01:00
*/
2017-02-21 17:33:07 +01:00
sostmt = castNode ( SetOperationStmt ,
transformSetOperationTree ( pstate , stmt , true , NULL ) ) ;
Assert ( sostmt ) ;
2000-11-05 01:15:54 +01:00
qry - > setOperations = ( Node * ) sostmt ;
2000-10-05 21:11:39 +02:00
/*
* Re - find leftmost SELECT ( now it ' s a sub - query in rangetable )
*/
2000-11-05 01:15:54 +01:00
node = sostmt - > larg ;
2000-10-05 21:11:39 +02:00
while ( node & & IsA ( node , SetOperationStmt ) )
node = ( ( SetOperationStmt * ) node ) - > larg ;
Assert ( node & & IsA ( node , RangeTblRef ) ) ;
2000-11-05 02:42:07 +01:00
leftmostRTI = ( ( RangeTblRef * ) node ) - > rtindex ;
leftmostQuery = rt_fetch ( leftmostRTI , pstate - > p_rtable ) - > subquery ;
2000-10-05 21:11:39 +02:00
Assert ( leftmostQuery ! = NULL ) ;
2001-03-22 05:01:46 +01:00
2000-10-05 21:11:39 +02:00
/*
* Generate dummy targetlist for outer query using column names of
2011-04-22 23:43:18 +02:00
* leftmost select and common datatypes / collations of topmost set
* operation . Also make lists of the dummy vars and their names for use
* in parsing ORDER BY .
2003-02-11 05:13:06 +01:00
*
* Note : we use leftmostRTI as the varno of the dummy variables . It
* shouldn ' t matter too much which RT index they have , as long as they
* have one that corresponds to a real RT entry ; else funny things may
* happen when the tree is mashed by rule rewriting .
2000-10-05 21:11:39 +02:00
*/
qry - > targetList = NIL ;
2002-04-28 21:54:29 +02:00
targetvars = NIL ;
2001-02-15 02:10:28 +01:00
targetnames = NIL ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
sortnscolumns = ( ParseNamespaceColumn * )
palloc0 ( list_length ( sostmt - > colTypes ) * sizeof ( ParseNamespaceColumn ) ) ;
sortcolindex = 0 ;
2004-05-26 06:41:50 +02:00
2019-02-28 20:25:01 +01:00
forfour ( lct , sostmt - > colTypes ,
lcm , sostmt - > colTypmods ,
lcc , sostmt - > colCollations ,
left_tlist , leftmostQuery - > targetList )
2000-10-05 21:11:39 +02:00
{
2006-08-10 04:36:29 +02:00
Oid colType = lfirst_oid ( lct ) ;
int32 colTypmod = lfirst_int ( lcm ) ;
2011-02-08 22:04:18 +01:00
Oid colCollation = lfirst_oid ( lcc ) ;
2005-04-06 18:34:07 +02:00
TargetEntry * lefttle = ( TargetEntry * ) lfirst ( left_tlist ) ;
2003-08-12 01:04:50 +02:00
char * colName ;
2005-04-06 18:34:07 +02:00
TargetEntry * tle ;
2008-10-04 23:56:55 +02:00
Var * var ;
2000-10-05 21:11:39 +02:00
2005-04-06 18:34:07 +02:00
Assert ( ! lefttle - > resjunk ) ;
colName = pstrdup ( lefttle - > resname ) ;
2008-10-04 23:56:55 +02:00
var = makeVar ( leftmostRTI ,
lefttle - > resno ,
colType ,
colTypmod ,
2011-02-08 22:04:18 +01:00
colCollation ,
2008-10-04 23:56:55 +02:00
0 ) ;
var - > location = exprLocation ( ( Node * ) lefttle - > expr ) ;
tle = makeTargetEntry ( ( Expr * ) var ,
2005-04-06 18:34:07 +02:00
( AttrNumber ) pstate - > p_next_resno + + ,
colName ,
false ) ;
qry - > targetList = lappend ( qry - > targetList , tle ) ;
2008-10-04 23:56:55 +02:00
targetvars = lappend ( targetvars , var ) ;
2001-02-15 02:10:28 +01:00
targetnames = lappend ( targetnames , makeString ( colName ) ) ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
sortnscolumns [ sortcolindex ] . p_varno = leftmostRTI ;
sortnscolumns [ sortcolindex ] . p_varattno = lefttle - > resno ;
sortnscolumns [ sortcolindex ] . p_vartype = colType ;
sortnscolumns [ sortcolindex ] . p_vartypmod = colTypmod ;
sortnscolumns [ sortcolindex ] . p_varcollid = colCollation ;
sortnscolumns [ sortcolindex ] . p_varnosyn = leftmostRTI ;
sortnscolumns [ sortcolindex ] . p_varattnosyn = lefttle - > resno ;
sortcolindex + + ;
2000-10-05 21:11:39 +02:00
}
2001-03-22 05:01:46 +01:00
2001-02-15 02:10:28 +01:00
/*
* As a first step towards supporting sort clauses that are expressions
2012-08-08 22:41:04 +02:00
* using the output columns , generate a namespace entry that makes the
2002-03-12 01:52:10 +01:00
* output columns visible . A Join RTE node is handy for this , since we
2001-02-15 02:10:28 +01:00
* can easily control the Vars generated upon matches .
*
* Note : we don ' t yet do anything useful with such cases , but at least
* " ORDER BY upper(foo) " will draw the right error message rather than
* " foo not found " .
*/
2009-01-22 21:16:10 +01:00
sv_rtable_length = list_length ( pstate - > p_rtable ) ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
jnsitem = addRangeTableEntryForJoin ( pstate ,
targetnames ,
sortnscolumns ,
JOIN_INNER ,
Reconsider the representation of join alias Vars.
The core idea of this patch is to make the parser generate join alias
Vars (that is, ones with varno pointing to a JOIN RTE) only when the
alias Var is actually different from any raw join input, that is a type
coercion and/or COALESCE is necessary to generate the join output value.
Otherwise just generate varno/varattno pointing to the relevant join
input column.
In effect, this means that the planner's flatten_join_alias_vars()
transformation is already done in the parser, for all cases except
(a) columns that are merged by JOIN USING and are transformed in the
process, and (b) whole-row join Vars. In principle that would allow
us to skip doing flatten_join_alias_vars() in many more queries than
we do now, but we don't have quite enough infrastructure to know that
we can do so --- in particular there's no cheap way to know whether
there are any whole-row join Vars. I'm not sure if it's worth the
trouble to add a Query-level flag for that, and in any case it seems
like fit material for a separate patch. But even without skipping the
work entirely, this should make flatten_join_alias_vars() faster,
particularly where there are nested joins that it previously had to
flatten recursively.
An essential part of this change is to replace Var nodes'
varnoold/varoattno fields with varnosyn/varattnosyn, which have
considerably more tightly-defined meanings than the old fields: when
they differ from varno/varattno, they identify the Var's position in
an aliased JOIN RTE, and the join alias is what ruleutils.c should
print for the Var. This is necessary because the varno change
destroyed ruleutils.c's ability to find the JOIN RTE from the Var's
varno.
Another way in which this change broke ruleutils.c is that it's no
longer feasible to determine, from a JOIN RTE's joinaliasvars list,
which join columns correspond to which columns of the join's immediate
input relations. (If those are sub-joins, the joinaliasvars entries
may point to columns of their base relations, not the sub-joins.)
But that was a horrid mess requiring a lot of fragile assumptions
already, so let's just bite the bullet and add some more JOIN RTE
fields to make it more straightforward to figure that out. I added
two integer-List fields containing the relevant column numbers from
the left and right input rels, plus a count of how many merged columns
there are.
This patch depends on the ParseNamespaceColumn infrastructure that
I added in commit 5815696bc. The biggest bit of code change is
restructuring transformFromClauseItem's handling of JOINs so that
the ParseNamespaceColumn data is propagated upward correctly.
Other than that and the ruleutils fixes, everything pretty much
just works, though some processing is now inessential. I grabbed
two pieces of low-hanging fruit in that line:
1. In find_expr_references, we don't need to recurse into join alias
Vars anymore. There aren't any except for references to merged USING
columns, which are more properly handled when we scan the join's RTE.
This change actually fixes an edge-case issue: we will now record a
dependency on any type-coercion function present in a USING column's
joinaliasvar, even if that join column has no references in the query
text. The odds of the missing dependency causing a problem seem quite
small: you'd have to posit somebody dropping an implicit cast between
two data types, without removing the types themselves, and then having
a stored rule containing a whole-row Var for a join whose USING merge
depends on that cast. So I don't feel a great need to change this in
the back branches. But in theory this way is more correct.
2. markRTEForSelectPriv and markTargetListOrigin don't need to recurse
into join alias Vars either, because the cases they care about don't
apply to alias Vars for USING columns that are semantically distinct
from the underlying columns. This removes the only case in which
markVarForSelectPriv could be called with NULL for the RTE, so adjust
the comments to describe that hack as being strictly internal to
markRTEForSelectPriv.
catversion bump required due to changes in stored rules.
Discussion: https://postgr.es/m/7115.1577986646@sss.pgh.pa.us
2020-01-09 17:56:59 +01:00
0 ,
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
targetvars ,
Reconsider the representation of join alias Vars.
The core idea of this patch is to make the parser generate join alias
Vars (that is, ones with varno pointing to a JOIN RTE) only when the
alias Var is actually different from any raw join input, that is a type
coercion and/or COALESCE is necessary to generate the join output value.
Otherwise just generate varno/varattno pointing to the relevant join
input column.
In effect, this means that the planner's flatten_join_alias_vars()
transformation is already done in the parser, for all cases except
(a) columns that are merged by JOIN USING and are transformed in the
process, and (b) whole-row join Vars. In principle that would allow
us to skip doing flatten_join_alias_vars() in many more queries than
we do now, but we don't have quite enough infrastructure to know that
we can do so --- in particular there's no cheap way to know whether
there are any whole-row join Vars. I'm not sure if it's worth the
trouble to add a Query-level flag for that, and in any case it seems
like fit material for a separate patch. But even without skipping the
work entirely, this should make flatten_join_alias_vars() faster,
particularly where there are nested joins that it previously had to
flatten recursively.
An essential part of this change is to replace Var nodes'
varnoold/varoattno fields with varnosyn/varattnosyn, which have
considerably more tightly-defined meanings than the old fields: when
they differ from varno/varattno, they identify the Var's position in
an aliased JOIN RTE, and the join alias is what ruleutils.c should
print for the Var. This is necessary because the varno change
destroyed ruleutils.c's ability to find the JOIN RTE from the Var's
varno.
Another way in which this change broke ruleutils.c is that it's no
longer feasible to determine, from a JOIN RTE's joinaliasvars list,
which join columns correspond to which columns of the join's immediate
input relations. (If those are sub-joins, the joinaliasvars entries
may point to columns of their base relations, not the sub-joins.)
But that was a horrid mess requiring a lot of fragile assumptions
already, so let's just bite the bullet and add some more JOIN RTE
fields to make it more straightforward to figure that out. I added
two integer-List fields containing the relevant column numbers from
the left and right input rels, plus a count of how many merged columns
there are.
This patch depends on the ParseNamespaceColumn infrastructure that
I added in commit 5815696bc. The biggest bit of code change is
restructuring transformFromClauseItem's handling of JOINs so that
the ParseNamespaceColumn data is propagated upward correctly.
Other than that and the ruleutils fixes, everything pretty much
just works, though some processing is now inessential. I grabbed
two pieces of low-hanging fruit in that line:
1. In find_expr_references, we don't need to recurse into join alias
Vars anymore. There aren't any except for references to merged USING
columns, which are more properly handled when we scan the join's RTE.
This change actually fixes an edge-case issue: we will now record a
dependency on any type-coercion function present in a USING column's
joinaliasvar, even if that join column has no references in the query
text. The odds of the missing dependency causing a problem seem quite
small: you'd have to posit somebody dropping an implicit cast between
two data types, without removing the types themselves, and then having
a stored rule containing a whole-row Var for a join whose USING merge
depends on that cast. So I don't feel a great need to change this in
the back branches. But in theory this way is more correct.
2. markRTEForSelectPriv and markTargetListOrigin don't need to recurse
into join alias Vars either, because the cases they care about don't
apply to alias Vars for USING columns that are semantically distinct
from the underlying columns. This removes the only case in which
markVarForSelectPriv could be called with NULL for the RTE, so adjust
the comments to describe that hack as being strictly internal to
markRTEForSelectPriv.
catversion bump required due to changes in stored rules.
Discussion: https://postgr.es/m/7115.1577986646@sss.pgh.pa.us
2020-01-09 17:56:59 +01:00
NIL ,
NIL ,
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
NULL ,
2021-03-31 17:09:24 +02:00
NULL ,
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
false ) ;
2002-03-12 01:52:10 +01:00
2012-08-08 22:41:04 +02:00
sv_namespace = pstate - > p_namespace ;
pstate - > p_namespace = NIL ;
2012-08-08 01:02:54 +02:00
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
/* add jnsitem to column namespace only */
addNSItemToQuery ( pstate , jnsitem , false , false , true ) ;
2001-02-15 02:10:28 +01:00
2000-10-05 21:11:39 +02:00
/*
* For now , we don ' t support resjunk sort clauses on the output of a
* setOperation tree - - - you can only use the SQL92 - spec options of
* selecting an output column by name or number . Enforce by checking that
* transformSortClause doesn ' t add any items to tlist .
*/
2004-05-31 01:40:41 +02:00
tllen = list_length ( qry - > targetList ) ;
2000-10-05 21:11:39 +02:00
qry - > sortClause = transformSortClause ( pstate ,
sortClause ,
2004-05-23 19:10:54 +02:00
& qry - > targetList ,
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
EXPR_KIND_ORDER_BY ,
2009-12-15 18:57:48 +01:00
false /* allow SQL92 rules */ ) ;
2000-10-05 21:11:39 +02:00
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
/* restore namespace, remove join RTE from rtable */
2012-08-08 22:41:04 +02:00
pstate - > p_namespace = sv_namespace ;
2009-01-22 21:16:10 +01:00
pstate - > p_rtable = list_truncate ( pstate - > p_rtable , sv_rtable_length ) ;
2001-02-15 02:10:28 +01:00
2004-05-31 01:40:41 +02:00
if ( tllen ! = list_length ( qry - > targetList ) )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2007-01-12 20:23:38 +01:00
errmsg ( " invalid UNION/INTERSECT/EXCEPT ORDER BY clause " ) ,
errdetail ( " Only result column names can be used, not expressions or functions. " ) ,
2008-09-01 22:42:46 +02:00
errhint ( " Add the expression/function to every SELECT, or move the UNION into a FROM clause. " ) ,
parser_errposition ( pstate ,
exprLocation ( list_nth ( qry - > targetList , tllen ) ) ) ) ) ;
2000-10-05 21:11:39 +02:00
2003-07-03 21:07:54 +02:00
qry - > limitOffset = transformLimitClause ( pstate , limitOffset ,
2020-04-07 22:22:13 +02:00
EXPR_KIND_OFFSET , " OFFSET " ,
stmt - > limitOption ) ;
2003-07-03 21:07:54 +02:00
qry - > limitCount = transformLimitClause ( pstate , limitCount ,
2020-04-07 22:22:13 +02:00
EXPR_KIND_LIMIT , " LIMIT " ,
stmt - > limitOption ) ;
qry - > limitOption = stmt - > limitOption ;
2000-10-05 21:11:39 +02:00
2003-01-17 04:25:04 +01:00
qry - > rtable = pstate - > p_rtable ;
qry - > jointree = makeFromExpr ( pstate - > p_joinlist , NULL ) ;
2000-10-05 21:11:39 +02:00
qry - > hasSubLinks = pstate - > p_hasSubLinks ;
2008-12-28 19:54:01 +01:00
qry - > hasWindowFuncs = pstate - > p_hasWindowFuncs ;
2016-09-13 19:54:24 +02:00
qry - > hasTargetSRFs = pstate - > p_hasTargetSRFs ;
2012-02-08 19:15:02 +01:00
qry - > hasAggs = pstate - > p_hasAggs ;
1998-07-19 07:49:26 +02:00
2006-04-30 20:30:40 +02:00
foreach ( l , lockingClause )
{
2009-10-28 15:55:47 +01:00
transformLockingClause ( pstate , qry ,
( LockingClause * ) lfirst ( l ) , false ) ;
2006-04-30 20:30:40 +02:00
}
1999-01-21 17:08:55 +01:00
2011-03-20 01:29:08 +01:00
assign_query_collations ( pstate , qry ) ;
2019-01-17 06:33:01 +01:00
/* this must be done after collations, for reliable comparison of exprs */
if ( pstate - > p_hasAggs | | qry - > groupClause | | qry - > groupingSets | | qry - > havingQual )
parseCheckAggregates ( pstate , qry ) ;
2000-10-05 21:11:39 +02:00
return qry ;
}
2021-02-01 13:54:59 +01:00
/*
* Make a SortGroupClause node for a SetOperationStmt ' s groupClauses
Disable anonymous record hash support except in special cases
Commit 01e658fa74 added hash support for row types. This also added
support for hashing anonymous record types, using the same approach
that the type cache uses for comparison support for record types: It
just reports that it works, but it might fail at run time if a
component type doesn't actually support the operation. We get away
with that for comparison because most types support that. But some
types don't support hashing, so the current state can result in
failures at run time where the planner chooses hashing over sorting,
whereas that previously worked if only sorting was an option.
We do, however, want the record hashing support for path tracking in
recursive unions, and the SEARCH and CYCLE clauses built on that. In
that case, hashing is the only plan option. So enable that, this
commit implements the following approach: The type cache does not
report that hashing is available for the record type. This undoes
that part of 01e658fa74. Instead, callers that require hashing no
matter what can override that result themselves. This patch only
touches the callers to make the aforementioned recursive query cases
work, namely the parse analysis of unions, as well as the hash_array()
function.
Reported-by: Sait Talha Nisanci <sait.nisanci@microsoft.com>
Bug: #17158
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
2021-09-08 09:25:46 +02:00
*
* If require_hash is true , the caller is indicating that they need hash
* support or they will fail . So look extra hard for hash support .
2021-02-01 13:54:59 +01:00
*/
SortGroupClause *
Disable anonymous record hash support except in special cases
Commit 01e658fa74 added hash support for row types. This also added
support for hashing anonymous record types, using the same approach
that the type cache uses for comparison support for record types: It
just reports that it works, but it might fail at run time if a
component type doesn't actually support the operation. We get away
with that for comparison because most types support that. But some
types don't support hashing, so the current state can result in
failures at run time where the planner chooses hashing over sorting,
whereas that previously worked if only sorting was an option.
We do, however, want the record hashing support for path tracking in
recursive unions, and the SEARCH and CYCLE clauses built on that. In
that case, hashing is the only plan option. So enable that, this
commit implements the following approach: The type cache does not
report that hashing is available for the record type. This undoes
that part of 01e658fa74. Instead, callers that require hashing no
matter what can override that result themselves. This patch only
touches the callers to make the aforementioned recursive query cases
work, namely the parse analysis of unions, as well as the hash_array()
function.
Reported-by: Sait Talha Nisanci <sait.nisanci@microsoft.com>
Bug: #17158
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
2021-09-08 09:25:46 +02:00
makeSortGroupClauseForSetOp ( Oid rescoltype , bool require_hash )
2021-02-01 13:54:59 +01:00
{
SortGroupClause * grpcl = makeNode ( SortGroupClause ) ;
Oid sortop ;
Oid eqop ;
bool hashable ;
/* determine the eqop and optional sortop */
get_sort_group_operators ( rescoltype ,
false , true , false ,
& sortop , & eqop , NULL ,
& hashable ) ;
Disable anonymous record hash support except in special cases
Commit 01e658fa74 added hash support for row types. This also added
support for hashing anonymous record types, using the same approach
that the type cache uses for comparison support for record types: It
just reports that it works, but it might fail at run time if a
component type doesn't actually support the operation. We get away
with that for comparison because most types support that. But some
types don't support hashing, so the current state can result in
failures at run time where the planner chooses hashing over sorting,
whereas that previously worked if only sorting was an option.
We do, however, want the record hashing support for path tracking in
recursive unions, and the SEARCH and CYCLE clauses built on that. In
that case, hashing is the only plan option. So enable that, this
commit implements the following approach: The type cache does not
report that hashing is available for the record type. This undoes
that part of 01e658fa74. Instead, callers that require hashing no
matter what can override that result themselves. This patch only
touches the callers to make the aforementioned recursive query cases
work, namely the parse analysis of unions, as well as the hash_array()
function.
Reported-by: Sait Talha Nisanci <sait.nisanci@microsoft.com>
Bug: #17158
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
2021-09-08 09:25:46 +02:00
/*
* The type cache doesn ' t believe that record is hashable ( see
* cache_record_field_properties ( ) ) , but if the caller really needs hash
* support , we can assume it does . Worst case , if any components of the
* record don ' t support hashing , we will fail at execution .
*/
if ( require_hash & & ( rescoltype = = RECORDOID | | rescoltype = = RECORDARRAYOID ) )
hashable = true ;
2021-02-01 13:54:59 +01:00
/* we don't have a tlist yet, so can't assign sortgrouprefs */
grpcl - > tleSortGroupRef = 0 ;
grpcl - > eqop = eqop ;
grpcl - > sortop = sortop ;
grpcl - > nulls_first = false ; /* OK with or without sortop */
grpcl - > hashable = hashable ;
return grpcl ;
}
2000-10-05 21:11:39 +02:00
/*
* transformSetOperationTree
* Recursively transform leaves and internal nodes of a set - op tree
2008-08-29 01:09:48 +02:00
*
2011-03-16 02:51:36 +01:00
* In addition to returning the transformed node , if targetlist isn ' t NULL
* then we return a list of its non - resjunk TargetEntry nodes . For a leaf
* set - op node these are the actual targetlist entries ; otherwise they are
* dummy entries created to carry the type , typmod , collation , and location
* ( for error messages ) of each output column of the set - op node . This info
* is needed only during the internal recursion of this function , so outside
* callers pass NULL for targetlist . Note : the reason for passing the
* actual targetlist entries of a leaf node is so that upper levels can
* replace UNKNOWN Consts with properly - coerced constants .
2000-10-05 21:11:39 +02:00
*/
static Node *
2008-08-29 01:09:48 +02:00
transformSetOperationTree ( ParseState * pstate , SelectStmt * stmt ,
2011-03-16 02:51:36 +01:00
bool isTopLevel , List * * targetlist )
2000-10-05 21:11:39 +02:00
{
2000-11-05 01:15:54 +01:00
bool isLeaf ;
Assert ( stmt & & IsA ( stmt , SelectStmt ) ) ;
2012-11-12 01:56:10 +01:00
/* Guard against stack overflow due to overly complex set-expressions */
check_stack_depth ( ) ;
2000-11-05 01:15:54 +01:00
/*
* Validity - check both leaf and internal SELECTs for disallowed ops .
*/
2007-04-28 00:05:49 +02:00
if ( stmt - > intoClause )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
2008-09-01 22:42:46 +02:00
errmsg ( " INTO is only allowed on first SELECT of UNION/INTERSECT/EXCEPT " ) ,
parser_errposition ( pstate ,
exprLocation ( ( Node * ) stmt - > intoClause ) ) ) ) ;
2005-04-28 23:47:18 +02:00
/* We don't support FOR UPDATE/SHARE with set ops at the moment. */
2005-08-01 22:31:16 +02:00
if ( stmt - > lockingClause )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s is not allowed with UNION/INTERSECT/EXCEPT " ,
LCS_asString ( ( ( LockingClause * )
linitial ( stmt - > lockingClause ) ) - > strength ) ) ) ) ;
2000-11-05 01:15:54 +01:00
/*
2012-07-31 23:56:21 +02:00
* If an internal node of a set - op tree has ORDER BY , LIMIT , FOR UPDATE ,
* or WITH clauses attached , we need to treat it like a leaf node to
2012-08-08 01:02:54 +02:00
* generate an independent sub - Query tree . Otherwise , it can be
2012-07-31 23:56:21 +02:00
* represented by a SetOperationStmt node underneath the parent Query .
2000-11-05 01:15:54 +01:00
*/
if ( stmt - > op = = SETOP_NONE )
{
Assert ( stmt - > larg = = NULL & & stmt - > rarg = = NULL ) ;
isLeaf = true ;
}
else
2000-10-05 21:11:39 +02:00
{
2000-11-05 01:15:54 +01:00
Assert ( stmt - > larg ! = NULL & & stmt - > rarg ! = NULL ) ;
if ( stmt - > sortClause | | stmt - > limitOffset | | stmt - > limitCount | |
2012-07-31 23:56:21 +02:00
stmt - > lockingClause | | stmt - > withClause )
2000-11-05 01:15:54 +01:00
isLeaf = true ;
else
isLeaf = false ;
}
if ( isLeaf )
{
/* Process leaf SELECT */
2000-10-05 21:11:39 +02:00
Query * selectQuery ;
char selectName [ 32 ] ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
ParseNamespaceItem * nsitem ;
2000-10-05 21:11:39 +02:00
RangeTblRef * rtr ;
2008-08-29 01:09:48 +02:00
ListCell * tl ;
2000-10-05 21:11:39 +02:00
/*
2001-02-14 22:35:07 +01:00
* Transform SelectStmt into a Query .
*
Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column). This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on. For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison". Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.
To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command. In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.
Also, change creation of an unknown-type column in a relation from a
warning to a hard error. The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE. We want to forbid that because
it's nothing but a foot-gun.
This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have. Add a checking pass to detect that. While at it,
we can detect tables or composite types that would fail, essentially
for free. Those would fail safely anyway later on, but we might as
well fail earlier.
This patch is by me, but it owes something to previous investigations
by Rahila Syed. Also thanks to Ashutosh Bapat and Michael Paquier for
review.
Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 15:17:18 +01:00
* This works the same as SELECT transformation normally would , except
* that we prevent resolving unknown - type outputs as TEXT . This does
* not change the subquery ' s semantics since if the column type
* matters semantically , it would have been resolved to something else
* anyway . Doing this lets us resolve such outputs using
* select_common_type ( ) , below .
*
2001-02-14 22:35:07 +01:00
* Note : previously transformed sub - queries don ' t affect the parsing
* of this sub - query , because they are not in the toplevel pstate ' s
* namespace list .
2000-10-05 21:11:39 +02:00
*/
Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column). This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on. For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison". Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.
To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command. In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.
Also, change creation of an unknown-type column in a relation from a
warning to a hard error. The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE. We want to forbid that because
it's nothing but a foot-gun.
This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have. Add a checking pass to detect that. While at it,
we can detect tables or composite types that would fail, essentially
for free. Those would fail safely anyway later on, but we might as
well fail earlier.
This patch is by me, but it owes something to previous investigations
by Rahila Syed. Also thanks to Ashutosh Bapat and Michael Paquier for
review.
Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 15:17:18 +01:00
selectQuery = parse_sub_analyze ( ( Node * ) stmt , pstate ,
NULL , false , false ) ;
2003-02-13 21:45:22 +01:00
/*
* Check for bogus references to Vars on the current query level ( but
* upper - level references are okay ) . Normally this can ' t happen
* because the namespace will be empty , but it could happen if we are
* inside a rule .
*/
2012-08-08 22:41:04 +02:00
if ( pstate - > p_namespace )
2003-02-13 21:45:22 +01:00
{
if ( contain_vars_of_level ( ( Node * ) selectQuery , 1 ) )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_INVALID_COLUMN_REFERENCE ) ,
2008-09-01 22:42:46 +02:00
errmsg ( " UNION/INTERSECT/EXCEPT member statement cannot refer to other relations of same query level " ) ,
parser_errposition ( pstate ,
locate_var_of_level ( ( Node * ) selectQuery , 1 ) ) ) ) ;
2003-02-13 21:45:22 +01:00
}
2001-03-22 05:01:46 +01:00
2008-08-29 01:09:48 +02:00
/*
2011-03-16 02:51:36 +01:00
* Extract a list of the non - junk TLEs for upper - level processing .
2008-08-29 01:09:48 +02:00
*/
2011-03-16 02:51:36 +01:00
if ( targetlist )
2008-08-29 01:09:48 +02:00
{
2011-03-16 02:51:36 +01:00
* targetlist = NIL ;
foreach ( tl , selectQuery - > targetList )
{
TargetEntry * tle = ( TargetEntry * ) lfirst ( tl ) ;
2008-08-29 01:09:48 +02:00
2011-03-16 02:51:36 +01:00
if ( ! tle - > resjunk )
* targetlist = lappend ( * targetlist , tle ) ;
}
2008-08-29 01:09:48 +02:00
}
2000-10-05 21:11:39 +02:00
/*
* Make the leaf query be a subquery in the top - level rangetable .
*/
2003-07-19 22:20:53 +02:00
snprintf ( selectName , sizeof ( selectName ) , " *SELECT* %d " ,
2004-05-31 01:40:41 +02:00
list_length ( pstate - > p_rtable ) + 1 ) ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
nsitem = addRangeTableEntryForSubquery ( pstate ,
selectQuery ,
makeAlias ( selectName , NIL ) ,
false ,
false ) ;
2001-03-22 05:01:46 +01:00
2000-10-05 21:11:39 +02:00
/*
* Return a RangeTblRef to replace the SelectStmt in the set - op tree .
*/
rtr = makeNode ( RangeTblRef ) ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
rtr - > rtindex = nsitem - > p_rtindex ;
2000-10-05 21:11:39 +02:00
return ( Node * ) rtr ;
}
2000-11-05 01:15:54 +01:00
else
2000-10-05 21:11:39 +02:00
{
2000-11-05 01:15:54 +01:00
/* Process an internal node (set operation node) */
SetOperationStmt * op = makeNode ( SetOperationStmt ) ;
2011-03-16 02:51:36 +01:00
List * ltargetlist ;
List * rtargetlist ;
ListCell * ltl ;
ListCell * rtl ;
2000-10-05 21:11:39 +02:00
const char * context ;
Disable anonymous record hash support except in special cases
Commit 01e658fa74 added hash support for row types. This also added
support for hashing anonymous record types, using the same approach
that the type cache uses for comparison support for record types: It
just reports that it works, but it might fail at run time if a
component type doesn't actually support the operation. We get away
with that for comparison because most types support that. But some
types don't support hashing, so the current state can result in
failures at run time where the planner chooses hashing over sorting,
whereas that previously worked if only sorting was an option.
We do, however, want the record hashing support for path tracking in
recursive unions, and the SEARCH and CYCLE clauses built on that. In
that case, hashing is the only plan option. So enable that, this
commit implements the following approach: The type cache does not
report that hashing is available for the record type. This undoes
that part of 01e658fa74. Instead, callers that require hashing no
matter what can override that result themselves. This patch only
touches the callers to make the aforementioned recursive query cases
work, namely the parse analysis of unions, as well as the hash_array()
function.
Reported-by: Sait Talha Nisanci <sait.nisanci@microsoft.com>
Bug: #17158
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
2021-09-08 09:25:46 +02:00
bool recursive = ( pstate - > p_parent_cte & &
pstate - > p_parent_cte - > cterecursive ) ;
2000-10-05 21:11:39 +02:00
2000-11-05 01:15:54 +01:00
context = ( stmt - > op = = SETOP_UNION ? " UNION " :
( stmt - > op = = SETOP_INTERSECT ? " INTERSECT " :
2000-10-05 21:11:39 +02:00
" EXCEPT " ) ) ;
2000-11-05 01:15:54 +01:00
op - > op = stmt - > op ;
op - > all = stmt - > all ;
2000-10-05 21:11:39 +02:00
/*
2009-09-09 05:32:52 +02:00
* Recursively transform the left child node .
2000-10-05 21:11:39 +02:00
*/
2008-08-29 01:09:48 +02:00
op - > larg = transformSetOperationTree ( pstate , stmt - > larg ,
2009-09-09 05:32:52 +02:00
false ,
2011-03-16 02:51:36 +01:00
& ltargetlist ) ;
2009-09-09 05:32:52 +02:00
/*
* If we are processing a recursive union query , now is the time to
* examine the non - recursive term ' s output columns and mark the
* containing CTE as having those result columns . We should do this
* only at the topmost setop of the CTE , of course .
*/
Disable anonymous record hash support except in special cases
Commit 01e658fa74 added hash support for row types. This also added
support for hashing anonymous record types, using the same approach
that the type cache uses for comparison support for record types: It
just reports that it works, but it might fail at run time if a
component type doesn't actually support the operation. We get away
with that for comparison because most types support that. But some
types don't support hashing, so the current state can result in
failures at run time where the planner chooses hashing over sorting,
whereas that previously worked if only sorting was an option.
We do, however, want the record hashing support for path tracking in
recursive unions, and the SEARCH and CYCLE clauses built on that. In
that case, hashing is the only plan option. So enable that, this
commit implements the following approach: The type cache does not
report that hashing is available for the record type. This undoes
that part of 01e658fa74. Instead, callers that require hashing no
matter what can override that result themselves. This patch only
touches the callers to make the aforementioned recursive query cases
work, namely the parse analysis of unions, as well as the hash_array()
function.
Reported-by: Sait Talha Nisanci <sait.nisanci@microsoft.com>
Bug: #17158
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
2021-09-08 09:25:46 +02:00
if ( isTopLevel & & recursive )
2011-03-16 02:51:36 +01:00
determineRecursiveColTypes ( pstate , op - > larg , ltargetlist ) ;
2009-09-09 05:32:52 +02:00
/*
* Recursively transform the right child node .
*/
2008-08-29 01:09:48 +02:00
op - > rarg = transformSetOperationTree ( pstate , stmt - > rarg ,
2009-09-09 05:32:52 +02:00
false ,
2011-03-16 02:51:36 +01:00
& rtargetlist ) ;
2001-03-22 05:01:46 +01:00
2000-10-05 21:11:39 +02:00
/*
* Verify that the two children have the same number of non - junk
* columns , and determine the types of the merged output columns .
*/
2011-03-16 02:51:36 +01:00
if ( list_length ( ltargetlist ) ! = list_length ( rtargetlist ) )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
errmsg ( " each %s query must have the same number of columns " ,
2008-08-29 01:09:48 +02:00
context ) ,
parser_errposition ( pstate ,
2011-03-16 02:51:36 +01:00
exprLocation ( ( Node * ) rtargetlist ) ) ) ) ;
2004-05-26 06:41:50 +02:00
2011-03-16 02:51:36 +01:00
if ( targetlist )
* targetlist = NIL ;
2000-10-05 21:11:39 +02:00
op - > colTypes = NIL ;
2006-08-10 04:36:29 +02:00
op - > colTypmods = NIL ;
2011-02-08 22:04:18 +01:00
op - > colCollations = NIL ;
2008-08-07 03:11:52 +02:00
op - > groupClauses = NIL ;
2011-03-16 02:51:36 +01:00
forboth ( ltl , ltargetlist , rtl , rtargetlist )
2000-10-05 21:11:39 +02:00
{
2011-03-16 02:51:36 +01:00
TargetEntry * ltle = ( TargetEntry * ) lfirst ( ltl ) ;
TargetEntry * rtle = ( TargetEntry * ) lfirst ( rtl ) ;
Node * lcolnode = ( Node * ) ltle - > expr ;
Node * rcolnode = ( Node * ) rtle - > expr ;
2009-12-16 23:24:13 +01:00
Oid lcoltype = exprType ( lcolnode ) ;
Oid rcoltype = exprType ( rcolnode ) ;
2008-08-29 01:09:48 +02:00
Node * bestexpr ;
2011-03-16 02:51:36 +01:00
int bestlocation ;
2000-10-05 21:11:39 +02:00
Oid rescoltype ;
2006-08-10 04:36:29 +02:00
int32 rescoltypmod ;
2011-02-08 22:04:18 +01:00
Oid rescolcoll ;
2000-10-05 21:11:39 +02:00
2006-08-10 04:36:29 +02:00
/* select common type, same as CASE et al */
2008-08-29 01:09:48 +02:00
rescoltype = select_common_type ( pstate ,
2009-12-16 23:24:13 +01:00
list_make2 ( lcolnode , rcolnode ) ,
2008-08-29 01:09:48 +02:00
context ,
& bestexpr ) ;
2011-03-16 02:51:36 +01:00
bestlocation = exprLocation ( bestexpr ) ;
2008-08-29 01:09:48 +02:00
2009-12-16 23:24:13 +01:00
/*
* Verify the coercions are actually possible . If not , we ' d fail
* later anyway , but we want to fail now while we have sufficient
* context to produce an error cursor position .
*
2011-03-16 02:51:36 +01:00
* For all non - UNKNOWN - type cases , we verify coercibility but we
* don ' t modify the child ' s expression , for fear of changing the
* child query ' s semantics .
*
* If a child expression is an UNKNOWN - type Const or Param , we
* want to replace it with the coerced expression . This can only
* happen when the child is a leaf set - op node . It ' s safe to
* replace the expression because if the child query ' s semantics
* depended on the type of this output column , it ' d have already
* coerced the UNKNOWN to something else . We want to do this
* because ( a ) we want to verify that a Const is valid for the
* target type , or resolve the actual type of an UNKNOWN Param ,
* and ( b ) we want to avoid unnecessary discrepancies between the
* output type of the child query and the resolved target type .
* Such a discrepancy would disable optimization in the planner .
*
2011-03-20 01:29:08 +01:00
* If it ' s some other UNKNOWN - type node , eg a Var , we do nothing
* ( knowing that coerce_to_common_type would fail ) . The planner
* is sometimes able to fold an UNKNOWN Var to a constant before
* it has to coerce the type , so failing now would just break
* cases that might work .
2009-12-16 23:24:13 +01:00
*/
2011-03-16 02:51:36 +01:00
if ( lcoltype ! = UNKNOWNOID )
2011-03-20 01:29:08 +01:00
lcolnode = coerce_to_common_type ( pstate , lcolnode ,
rescoltype , context ) ;
else if ( IsA ( lcolnode , Const ) | |
IsA ( lcolnode , Param ) )
{
lcolnode = coerce_to_common_type ( pstate , lcolnode ,
rescoltype , context ) ;
ltle - > expr = ( Expr * ) lcolnode ;
}
2011-03-16 02:51:36 +01:00
if ( rcoltype ! = UNKNOWNOID )
2011-03-20 01:29:08 +01:00
rcolnode = coerce_to_common_type ( pstate , rcolnode ,
rescoltype , context ) ;
else if ( IsA ( rcolnode , Const ) | |
IsA ( rcolnode , Param ) )
{
rcolnode = coerce_to_common_type ( pstate , rcolnode ,
rescoltype , context ) ;
rtle - > expr = ( Expr * ) rcolnode ;
}
2020-10-27 17:39:23 +01:00
rescoltypmod = select_common_typmod ( pstate ,
list_make2 ( lcolnode , rcolnode ) ,
rescoltype ) ;
2011-03-20 01:29:08 +01:00
/*
* Select common collation . A common collation is required for
* all set operators except UNION ALL ; see SQL : 2008 7.13 < query
* expression > Syntax Rule 15 c . ( If we fail to identify a common
2019-07-01 03:00:23 +02:00
* collation for a UNION ALL column , the colCollations element
2011-03-20 01:29:08 +01:00
* will be set to InvalidOid , which may result in a runtime error
* if something at a higher query level wants to use the column ' s
* collation . )
*/
rescolcoll = select_common_collation ( pstate ,
list_make2 ( lcolnode , rcolnode ) ,
( op - > op = = SETOP_UNION & & op - > all ) ) ;
2008-08-29 01:09:48 +02:00
/* emit results */
2004-05-31 01:40:41 +02:00
op - > colTypes = lappend_oid ( op - > colTypes , rescoltype ) ;
2006-08-10 04:36:29 +02:00
op - > colTypmods = lappend_int ( op - > colTypmods , rescoltypmod ) ;
2011-02-08 22:04:18 +01:00
op - > colCollations = lappend_oid ( op - > colCollations , rescolcoll ) ;
2006-08-10 04:36:29 +02:00
2008-08-07 03:11:52 +02:00
/*
* For all cases except UNION ALL , identify the grouping operators
* ( and , if available , sorting operators ) that will be used to
* eliminate duplicates .
*/
if ( op - > op ! = SETOP_UNION | | ! op - > all )
{
2008-09-01 22:42:46 +02:00
ParseCallbackState pcbstate ;
setup_parser_errposition_callback ( & pcbstate , pstate ,
2011-03-16 02:51:36 +01:00
bestlocation ) ;
2008-08-07 03:11:52 +02:00
Disable anonymous record hash support except in special cases
Commit 01e658fa74 added hash support for row types. This also added
support for hashing anonymous record types, using the same approach
that the type cache uses for comparison support for record types: It
just reports that it works, but it might fail at run time if a
component type doesn't actually support the operation. We get away
with that for comparison because most types support that. But some
types don't support hashing, so the current state can result in
failures at run time where the planner chooses hashing over sorting,
whereas that previously worked if only sorting was an option.
We do, however, want the record hashing support for path tracking in
recursive unions, and the SEARCH and CYCLE clauses built on that. In
that case, hashing is the only plan option. So enable that, this
commit implements the following approach: The type cache does not
report that hashing is available for the record type. This undoes
that part of 01e658fa74. Instead, callers that require hashing no
matter what can override that result themselves. This patch only
touches the callers to make the aforementioned recursive query cases
work, namely the parse analysis of unions, as well as the hash_array()
function.
Reported-by: Sait Talha Nisanci <sait.nisanci@microsoft.com>
Bug: #17158
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
2021-09-08 09:25:46 +02:00
/* If it's a recursive union, we need to require hashing support. */
2021-02-01 13:54:59 +01:00
op - > groupClauses = lappend ( op - > groupClauses ,
Disable anonymous record hash support except in special cases
Commit 01e658fa74 added hash support for row types. This also added
support for hashing anonymous record types, using the same approach
that the type cache uses for comparison support for record types: It
just reports that it works, but it might fail at run time if a
component type doesn't actually support the operation. We get away
with that for comparison because most types support that. But some
types don't support hashing, so the current state can result in
failures at run time where the planner chooses hashing over sorting,
whereas that previously worked if only sorting was an option.
We do, however, want the record hashing support for path tracking in
recursive unions, and the SEARCH and CYCLE clauses built on that. In
that case, hashing is the only plan option. So enable that, this
commit implements the following approach: The type cache does not
report that hashing is available for the record type. This undoes
that part of 01e658fa74. Instead, callers that require hashing no
matter what can override that result themselves. This patch only
touches the callers to make the aforementioned recursive query cases
work, namely the parse analysis of unions, as well as the hash_array()
function.
Reported-by: Sait Talha Nisanci <sait.nisanci@microsoft.com>
Bug: #17158
Discussion: https://www.postgresql.org/message-id/flat/17158-8a2ba823982537a4%40postgresql.org
2021-09-08 09:25:46 +02:00
makeSortGroupClauseForSetOp ( rescoltype , recursive ) ) ;
2008-08-07 03:11:52 +02:00
2008-09-01 22:42:46 +02:00
cancel_parser_errposition_callback ( & pcbstate ) ;
2008-08-07 03:11:52 +02:00
}
2011-03-16 02:51:36 +01:00
/*
* Construct a dummy tlist entry to return . We use a SetToDefault
* node for the expression , since it carries exactly the fields
* needed , but any other expression node type would do as well .
*/
if ( targetlist )
{
SetToDefault * rescolnode = makeNode ( SetToDefault ) ;
TargetEntry * restle ;
rescolnode - > typeId = rescoltype ;
rescolnode - > typeMod = rescoltypmod ;
2011-03-20 01:29:08 +01:00
rescolnode - > collation = rescolcoll ;
2011-03-16 02:51:36 +01:00
rescolnode - > location = bestlocation ;
restle = makeTargetEntry ( ( Expr * ) rescolnode ,
0 , /* no need to set resno */
NULL ,
false ) ;
* targetlist = lappend ( * targetlist , restle ) ;
}
2000-10-05 21:11:39 +02:00
}
2000-11-05 01:15:54 +01:00
2000-10-05 21:11:39 +02:00
return ( Node * ) op ;
}
1996-07-09 08:22:35 +02:00
}
2009-09-09 05:32:52 +02:00
/*
* Process the outputs of the non - recursive term of a recursive union
* to set up the parent CTE ' s columns
*/
static void
2011-03-16 02:51:36 +01:00
determineRecursiveColTypes ( ParseState * pstate , Node * larg , List * nrtargetlist )
2009-09-09 05:32:52 +02:00
{
Node * node ;
int leftmostRTI ;
Query * leftmostQuery ;
List * targetList ;
ListCell * left_tlist ;
2011-03-16 02:51:36 +01:00
ListCell * nrtl ;
2009-09-09 05:32:52 +02:00
int next_resno ;
/*
* Find leftmost leaf SELECT
*/
node = larg ;
while ( node & & IsA ( node , SetOperationStmt ) )
node = ( ( SetOperationStmt * ) node ) - > larg ;
Assert ( node & & IsA ( node , RangeTblRef ) ) ;
leftmostRTI = ( ( RangeTblRef * ) node ) - > rtindex ;
leftmostQuery = rt_fetch ( leftmostRTI , pstate - > p_rtable ) - > subquery ;
Assert ( leftmostQuery ! = NULL ) ;
/*
* Generate dummy targetlist using column names of leftmost select and
* dummy result expressions of the non - recursive term .
*/
targetList = NIL ;
next_resno = 1 ;
2019-02-28 20:25:01 +01:00
forboth ( nrtl , nrtargetlist , left_tlist , leftmostQuery - > targetList )
2009-09-09 05:32:52 +02:00
{
2011-03-16 02:51:36 +01:00
TargetEntry * nrtle = ( TargetEntry * ) lfirst ( nrtl ) ;
2009-09-09 05:32:52 +02:00
TargetEntry * lefttle = ( TargetEntry * ) lfirst ( left_tlist ) ;
char * colName ;
TargetEntry * tle ;
Assert ( ! lefttle - > resjunk ) ;
colName = pstrdup ( lefttle - > resname ) ;
2011-03-16 02:51:36 +01:00
tle = makeTargetEntry ( nrtle - > expr ,
2009-09-09 05:32:52 +02:00
next_resno + + ,
colName ,
false ) ;
targetList = lappend ( targetList , tle ) ;
}
/* Now build CTE's output column info using dummy targetlist */
analyzeCTETargetList ( pstate , pstate - > p_parent_cte , targetList ) ;
}
2000-10-05 21:11:39 +02:00
2021-04-07 21:30:08 +02:00
/*
* transformReturnStmt -
* transforms a return statement
*/
static Query *
transformReturnStmt ( ParseState * pstate , ReturnStmt * stmt )
{
Query * qry = makeNode ( Query ) ;
qry - > commandType = CMD_SELECT ;
qry - > isReturn = true ;
qry - > targetList = list_make1 ( makeTargetEntry ( ( Expr * ) transformExpr ( pstate , stmt - > returnval , EXPR_KIND_SELECT_TARGET ) ,
1 , NULL , false ) ) ;
if ( pstate - > p_resolve_unknowns )
resolveTargetListUnknowns ( pstate , qry - > targetList ) ;
qry - > rtable = pstate - > p_rtable ;
qry - > jointree = makeFromExpr ( pstate - > p_joinlist , NULL ) ;
qry - > hasSubLinks = pstate - > p_hasSubLinks ;
qry - > hasWindowFuncs = pstate - > p_hasWindowFuncs ;
qry - > hasTargetSRFs = pstate - > p_hasTargetSRFs ;
qry - > hasAggs = pstate - > p_hasAggs ;
assign_query_collations ( pstate , qry ) ;
return qry ;
}
1996-07-09 08:22:35 +02:00
/*
* transformUpdateStmt -
* transforms an update statement
*/
static Query *
1998-01-09 21:06:08 +01:00
transformUpdateStmt ( ParseState * pstate , UpdateStmt * stmt )
1996-07-09 08:22:35 +02:00
{
Query * qry = makeNode ( Query ) ;
2014-01-07 21:25:16 +01:00
ParseNamespaceItem * nsitem ;
2000-09-29 20:21:41 +02:00
Node * qual ;
1997-09-07 07:04:48 +02:00
1996-07-09 08:22:35 +02:00
qry - > commandType = CMD_UPDATE ;
2015-07-24 11:48:53 +02:00
pstate - > p_is_insert = false ;
1996-07-09 08:22:35 +02:00
2010-10-16 01:53:59 +02:00
/* process the WITH clause independently of all else */
if ( stmt - > withClause )
{
qry - > hasRecursive = stmt - > withClause - > recursive ;
qry - > cteList = transformWithClause ( pstate , stmt - > withClause ) ;
2011-02-26 00:56:23 +01:00
qry - > hasModifyingCTE = pstate - > p_hasModifyingCTE ;
2010-10-16 01:53:59 +02:00
}
2002-03-22 03:56:37 +01:00
qry - > resultRelation = setTargetTable ( pstate , stmt - > relation ,
2016-12-23 19:35:11 +01:00
stmt - > relation - > inh ,
2004-01-15 00:01:55 +01:00
true ,
ACL_UPDATE ) ;
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
nsitem = pstate - > p_target_nsitem ;
2014-01-07 21:25:16 +01:00
2014-01-12 01:03:12 +01:00
/* subqueries in FROM cannot access the result relation */
2014-01-07 21:25:16 +01:00
nsitem - > p_lateral_only = true ;
2014-01-12 01:03:12 +01:00
nsitem - > p_lateral_ok = false ;
2014-01-07 21:25:16 +01:00
1996-07-09 08:22:35 +02:00
/*
* the FROM clause is non - standard SQL syntax . We used to be able to do
* this with REPLACE in POSTQUEL so we keep the feature .
*/
2001-02-14 22:35:07 +01:00
transformFromClause ( pstate , stmt - > fromClause ) ;
1996-07-09 08:22:35 +02:00
2014-01-12 01:03:12 +01:00
/* remaining clauses can reference the result relation normally */
2014-01-07 21:25:16 +01:00
nsitem - > p_lateral_only = false ;
2014-01-12 01:03:12 +01:00
nsitem - > p_lateral_ok = true ;
2014-01-07 21:25:16 +01:00
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
qual = transformWhereClause ( pstate , stmt - > whereClause ,
EXPR_KIND_WHERE , " WHERE " ) ;
1999-07-19 02:26:20 +02:00
2006-08-12 04:52:06 +02:00
qry - > returningList = transformReturningList ( pstate , stmt - > returningList ) ;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
/*
* Now we are done with SELECT - like processing , and can get on with
* transforming the target list to match the UPDATE target columns .
*/
qry - > targetList = transformUpdateTargetList ( pstate , stmt - > targetList ) ;
1996-07-09 08:22:35 +02:00
qry - > rtable = pstate - > p_rtable ;
2000-09-29 20:21:41 +02:00
qry - > jointree = makeFromExpr ( pstate - > p_joinlist , qual ) ;
1998-01-10 05:30:11 +01:00
2016-09-13 19:54:24 +02:00
qry - > hasTargetSRFs = pstate - > p_hasTargetSRFs ;
2000-09-12 23:07:18 +02:00
qry - > hasSubLinks = pstate - > p_hasSubLinks ;
2006-06-21 20:30:11 +02:00
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
assign_query_collations ( pstate , qry ) ;
return qry ;
}
/*
* transformUpdateTargetList -
2018-04-12 12:22:56 +02:00
* handle SET clause in UPDATE / INSERT . . . ON CONFLICT UPDATE
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
*/
2018-04-12 12:22:56 +02:00
static List *
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
transformUpdateTargetList ( ParseState * pstate , List * origTlist )
{
List * tlist = NIL ;
RangeTblEntry * target_rte ;
ListCell * orig_tl ;
ListCell * tl ;
tlist = transformTargetList ( pstate , origTlist ,
EXPR_KIND_UPDATE_SOURCE ) ;
1999-07-19 02:26:20 +02:00
/* Prepare to assign non-conflicting resnos to resjunk attributes */
2018-04-07 22:00:39 +02:00
if ( pstate - > p_next_resno < = RelationGetNumberOfAttributes ( pstate - > p_target_relation ) )
pstate - > p_next_resno = RelationGetNumberOfAttributes ( pstate - > p_target_relation ) + 1 ;
1999-07-19 02:26:20 +02:00
/* Prepare non-junk columns for assignment to target table */
Make parser rely more heavily on the ParseNamespaceItem data structure.
When I added the ParseNamespaceItem data structure (in commit 5ebaaa494),
it wasn't very tightly integrated into the parser's APIs. In the wake of
adding p_rtindex to that struct (commit b541e9acc), there is a good reason
to make more use of it: by passing around ParseNamespaceItem pointers
instead of bare RTE pointers, we can get rid of various messy methods for
passing back or deducing the rangetable index of an RTE during parsing.
Hence, refactor the addRangeTableEntryXXX functions to build and return
a ParseNamespaceItem struct, not just the RTE proper; and replace
addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem
rather than building one internally.
Also, add per-column data (a ParseNamespaceColumn array) to each
ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX,
where we have column type data at hand so that it's nearly free to fill
the data structure. Later, when we need to build Vars referencing RTEs,
we can use the ParseNamespaceColumn info to avoid the rather expensive
operations done in get_rte_attribute_type() or expandRTE().
get_rte_attribute_type() is indeed dead code now, so I've removed it.
This makes for a useful improvement in parse analysis speed, around 20%
in one moderately-complex test query.
The ParseNamespaceColumn structs also include Var identity information
(varno/varattno). That info isn't actually being used in this patch,
except that p_varno == 0 is a handy test for a dropped column.
A follow-on patch will make more use of it.
Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
target_rte = pstate - > p_target_nsitem - > p_rte ;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
orig_tl = list_head ( origTlist ) ;
2004-05-26 06:41:50 +02:00
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
foreach ( tl , tlist )
1999-07-19 02:26:20 +02:00
{
TargetEntry * tle = ( TargetEntry * ) lfirst ( tl ) ;
ResTarget * origTarget ;
2006-03-23 01:19:30 +01:00
int attrno ;
1999-07-19 02:26:20 +02:00
2005-04-06 18:34:07 +02:00
if ( tle - > resjunk )
1999-07-19 02:26:20 +02:00
{
/*
* Resjunk nodes need no additional processing , but be sure they
2003-08-12 01:04:50 +02:00
* have resnos that do not match any target columns ; else rewriter
* or planner might get confused . They don ' t need a resname
* either .
1999-07-19 02:26:20 +02:00
*/
2005-04-06 18:34:07 +02:00
tle - > resno = ( AttrNumber ) pstate - > p_next_resno + + ;
tle - > resname = NULL ;
1999-07-19 02:26:20 +02:00
continue ;
}
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
if ( orig_tl = = NULL )
1999-07-19 02:26:20 +02:00
elog ( ERROR , " UPDATE target count mismatch --- internal error " ) ;
Improve castNode notation by introducing list-extraction-specific variants.
This extends the castNode() notation introduced by commit 5bcab1114 to
provide, in one step, extraction of a list cell's pointer and coercion to
a concrete node type. For example, "lfirst_node(Foo, lc)" is the same
as "castNode(Foo, lfirst(lc))". Almost half of the uses of castNode
that have appeared so far include a list extraction call, so this is
pretty widely useful, and it saves a few more keystrokes compared to the
old way.
As with the previous patch, back-patch the addition of these macros to
pg_list.h, so that the notation will be available when back-patching.
Patch by me, after an idea of Andrew Gierth's.
Discussion: https://postgr.es/m/14197.1491841216@sss.pgh.pa.us
2017-04-10 19:51:29 +02:00
origTarget = lfirst_node ( ResTarget , orig_tl ) ;
2003-06-25 06:19:24 +02:00
2006-03-23 01:19:30 +01:00
attrno = attnameAttNum ( pstate - > p_target_relation ,
origTarget - > name , true ) ;
if ( attrno = = InvalidAttrNumber )
ereport ( ERROR ,
( errcode ( ERRCODE_UNDEFINED_COLUMN ) ,
errmsg ( " column \" %s \" of relation \" %s \" does not exist " ,
origTarget - > name ,
RelationGetRelationName ( pstate - > p_target_relation ) ) ,
parser_errposition ( pstate , origTarget - > location ) ) ) ;
1999-11-01 06:06:21 +01:00
updateTargetListEntry ( pstate , tle , origTarget - > name ,
2006-03-23 01:19:30 +01:00
attrno ,
origTarget - > indirection ,
origTarget - > location ) ;
2003-06-25 06:19:24 +02:00
2009-01-22 21:16:10 +01:00
/* Mark the target column as requiring update permissions */
2015-05-08 00:20:46 +02:00
target_rte - > updatedCols = bms_add_member ( target_rte - > updatedCols ,
2009-01-22 21:16:10 +01:00
attrno - FirstLowInvalidHeapAttributeNumber ) ;
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
orig_tl = lnext ( origTlist , orig_tl ) ;
1999-07-19 02:26:20 +02:00
}
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
if ( orig_tl ! = NULL )
1999-07-19 02:26:20 +02:00
elog ( ERROR , " UPDATE target count mismatch --- internal error " ) ;
2020-02-17 15:19:58 +01:00
return tlist ;
}
2000-02-04 19:49:34 +01:00
/*
2006-08-12 04:52:06 +02:00
* transformReturningList -
* handle a RETURNING clause in INSERT / UPDATE / DELETE
*/
static List *
transformReturningList ( ParseState * pstate , List * returningList )
{
List * rlist ;
int save_next_resno ;
if ( returningList = = NIL )
return NIL ; /* nothing to do */
/*
* We need to assign resnos starting at one in the RETURNING list . Save
* and restore the main tlist ' s value of p_next_resno , just in case
* someone looks at it later ( probably won ' t happen ) .
*/
save_next_resno = pstate - > p_next_resno ;
pstate - > p_next_resno = 1 ;
/* transform RETURNING identically to a SELECT targetlist */
Centralize the logic for detecting misplaced aggregates, window funcs, etc.
Formerly we relied on checking after-the-fact to see if an expression
contained aggregates, window functions, or sub-selects when it shouldn't.
This is grotty, easily forgotten (indeed, we had forgotten to teach
DefineIndex about rejecting window functions), and none too efficient
since it requires extra traversals of the parse tree. To improve matters,
define an enum type that classifies all SQL sub-expressions, store it in
ParseState to show what kind of expression we are currently parsing, and
make transformAggregateCall, transformWindowFuncCall, and transformSubLink
check the expression type and throw error if the type indicates the
construct is disallowed. This allows removal of a large number of ad-hoc
checks scattered around the code base. The enum type is sufficiently
fine-grained that we can still produce error messages of at least the
same specificity as before.
Bringing these error checks together revealed that we'd been none too
consistent about phrasing of the error messages, so standardize the wording
a bit.
Also, rewrite checking of aggregate arguments so that it requires only one
traversal of the arguments, rather than up to three as before.
In passing, clean up some more comments left over from add_missing_from
support, and annotate some tests that I think are dead code now that that's
gone. (I didn't risk actually removing said dead code, though.)
2012-08-10 17:35:33 +02:00
rlist = transformTargetList ( pstate , returningList , EXPR_KIND_RETURNING ) ;
2006-08-12 04:52:06 +02:00
2013-12-15 02:23:26 +01:00
/*
* Complain if the nonempty tlist expanded to nothing ( which is possible
* if it contains only a star - expansion of a zero - column table ) . If we
* allow this , the parsed Query will look like it didn ' t have RETURNING ,
* with results that would probably surprise the user .
*/
if ( rlist = = NIL )
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
errmsg ( " RETURNING must have at least one column " ) ,
parser_errposition ( pstate ,
exprLocation ( linitial ( returningList ) ) ) ) ) ;
2006-08-12 04:52:06 +02:00
/* mark column origins */
markTargetListOrigins ( pstate , rlist ) ;
Change unknown-type literals to type text in SELECT and RETURNING lists.
Previously, we left such literals alone if the query or subquery had
no properties forcing a type decision to be made (such as an ORDER BY or
DISTINCT clause using that output column). This meant that "unknown" could
be an exposed output column type, which has never been a great idea because
it could result in strange failures later on. For example, an outer query
that tried to do any operations on an unknown-type subquery output would
generally fail with some weird error like "failed to find conversion
function from unknown to text" or "could not determine which collation to
use for string comparison". Also, if the case occurred in a CREATE VIEW's
query then the view would have an unknown-type column, causing similar
failures in queries trying to use the view.
To fix, at the tail end of parse analysis of a query, forcibly convert any
remaining "unknown" literals in its SELECT or RETURNING list to type text.
However, provide a switch to suppress that, and use it in the cases of
SELECT inside a set operation or INSERT command. In those cases we already
had type resolution rules that make use of context information from outside
the subquery proper, and we don't want to change that behavior.
Also, change creation of an unknown-type column in a relation from a
warning to a hard error. The error should be unreachable now in CREATE
VIEW or CREATE MATVIEW, but it's still possible to explicitly say "unknown"
in CREATE TABLE or CREATE (composite) TYPE. We want to forbid that because
it's nothing but a foot-gun.
This change creates a pg_upgrade failure case: a matview that contains an
unknown-type column can't be pg_upgraded, because reparsing the matview's
defining query will now decide that the column is of type text, which
doesn't match the cstring-like storage that the old materialized column
would actually have. Add a checking pass to detect that. While at it,
we can detect tables or composite types that would fail, essentially
for free. Those would fail safely anyway later on, but we might as
well fail earlier.
This patch is by me, but it owes something to previous investigations
by Rahila Syed. Also thanks to Ashutosh Bapat and Michael Paquier for
review.
Discussion: https://postgr.es/m/CAH2L28uwwbL9HUM-WR=hromW1Cvamkn7O-g8fPY2m=_7muJ0oA@mail.gmail.com
2017-01-25 15:17:18 +01:00
/* resolve any still-unresolved output columns as being type text */
if ( pstate - > p_resolve_unknowns )
resolveTargetListUnknowns ( pstate , rlist ) ;
2006-08-12 04:52:06 +02:00
/* restore state */
pstate - > p_next_resno = save_next_resno ;
return rlist ;
}
2002-08-27 06:55:12 +02:00
2021-01-04 17:52:00 +01:00
/*
* transformPLAssignStmt -
* transform a PL / pgSQL assignment statement
*
* If there is no opt_indirection , the transformed statement looks like
* " SELECT a_expr ... " , except the expression has been cast to the type of
* the target . With indirection , it ' s still a SELECT , but the expression will
* incorporate FieldStore and / or assignment SubscriptingRef nodes to compute a
* new value for a container - type variable represented by the target . The
* expression references the target as the container source .
*/
static Query *
transformPLAssignStmt ( ParseState * pstate , PLAssignStmt * stmt )
{
Query * qry = makeNode ( Query ) ;
ColumnRef * cref = makeNode ( ColumnRef ) ;
List * indirection = stmt - > indirection ;
int nnames = stmt - > nnames ;
SelectStmt * sstmt = stmt - > val ;
Node * target ;
Oid targettype ;
int32 targettypmod ;
Oid targetcollation ;
List * tlist ;
TargetEntry * tle ;
Oid type_id ;
Node * qual ;
ListCell * l ;
/*
* First , construct a ColumnRef for the target variable . If the target
* has more than one dotted name , we have to pull the extra names out of
* the indirection list .
*/
cref - > fields = list_make1 ( makeString ( stmt - > name ) ) ;
cref - > location = stmt - > location ;
if ( nnames > 1 )
{
/* avoid munging the raw parsetree */
indirection = list_copy ( indirection ) ;
while ( - - nnames > 0 & & indirection ! = NIL )
{
Node * ind = ( Node * ) linitial ( indirection ) ;
if ( ! IsA ( ind , String ) )
elog ( ERROR , " invalid name count in PLAssignStmt " ) ;
cref - > fields = lappend ( cref - > fields , ind ) ;
indirection = list_delete_first ( indirection ) ;
}
}
/*
* Transform the target reference . Typically we will get back a Param
* node , but there ' s no reason to be too picky about its type .
*/
target = transformExpr ( pstate , ( Node * ) cref ,
EXPR_KIND_UPDATE_TARGET ) ;
targettype = exprType ( target ) ;
targettypmod = exprTypmod ( target ) ;
targetcollation = exprCollation ( target ) ;
/*
* The rest mostly matches transformSelectStmt , except that we needn ' t
2021-01-22 22:26:22 +01:00
* consider WITH or INTO , and we build a targetlist our own way .
2021-01-04 17:52:00 +01:00
*/
qry - > commandType = CMD_SELECT ;
pstate - > p_is_insert = false ;
/* make FOR UPDATE/FOR SHARE info available to addRangeTableEntry */
pstate - > p_locking_clause = sstmt - > lockingClause ;
/* make WINDOW info available for window functions, too */
pstate - > p_windowdefs = sstmt - > windowClause ;
/* process the FROM clause */
transformFromClause ( pstate , sstmt - > fromClause ) ;
/* initially transform the targetlist as if in SELECT */
tlist = transformTargetList ( pstate , sstmt - > targetList ,
EXPR_KIND_SELECT_TARGET ) ;
/* we should have exactly one targetlist item */
if ( list_length ( tlist ) ! = 1 )
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
errmsg_plural ( " assignment source returned %d column " ,
" assignment source returned %d columns " ,
list_length ( tlist ) ,
list_length ( tlist ) ) ) ) ;
tle = linitial_node ( TargetEntry , tlist ) ;
/*
* This next bit is similar to transformAssignedExpr ; the key difference
* is we use COERCION_PLPGSQL not COERCION_ASSIGNMENT .
*/
type_id = exprType ( ( Node * ) tle - > expr ) ;
pstate - > p_expr_kind = EXPR_KIND_UPDATE_TARGET ;
if ( indirection )
{
tle - > expr = ( Expr * )
transformAssignmentIndirection ( pstate ,
target ,
stmt - > name ,
false ,
targettype ,
targettypmod ,
targetcollation ,
indirection ,
list_head ( indirection ) ,
( Node * ) tle - > expr ,
COERCION_PLPGSQL ,
exprLocation ( target ) ) ;
}
else if ( targettype ! = type_id & &
( targettype = = RECORDOID | | ISCOMPLEX ( targettype ) ) & &
( type_id = = RECORDOID | | ISCOMPLEX ( type_id ) ) )
{
/*
* Hack : do not let coerce_to_target_type ( ) deal with inconsistent
* composite types . Just pass the expression result through as - is ,
* and let the PL / pgSQL executor do the conversion its way . This is
* rather bogus , but it ' s needed for backwards compatibility .
*/
}
else
{
/*
* For normal non - qualified target column , do type checking and
* coercion .
*/
Node * orig_expr = ( Node * ) tle - > expr ;
tle - > expr = ( Expr * )
coerce_to_target_type ( pstate ,
orig_expr , type_id ,
targettype , targettypmod ,
COERCION_PLPGSQL ,
COERCE_IMPLICIT_CAST ,
- 1 ) ;
/* With COERCION_PLPGSQL, this error is probably unreachable */
if ( tle - > expr = = NULL )
ereport ( ERROR ,
( errcode ( ERRCODE_DATATYPE_MISMATCH ) ,
errmsg ( " variable \" %s \" is of type %s "
" but expression is of type %s " ,
stmt - > name ,
format_type_be ( targettype ) ,
format_type_be ( type_id ) ) ,
errhint ( " You will need to rewrite or cast the expression. " ) ,
parser_errposition ( pstate , exprLocation ( orig_expr ) ) ) ) ;
}
pstate - > p_expr_kind = EXPR_KIND_NONE ;
qry - > targetList = list_make1 ( tle ) ;
/* transform WHERE */
qual = transformWhereClause ( pstate , sstmt - > whereClause ,
EXPR_KIND_WHERE , " WHERE " ) ;
/* initial processing of HAVING clause is much like WHERE clause */
qry - > havingQual = transformWhereClause ( pstate , sstmt - > havingClause ,
EXPR_KIND_HAVING , " HAVING " ) ;
/*
* Transform sorting / grouping stuff . Do ORDER BY first because both
* transformGroupClause and transformDistinctClause need the results . Note
* that these functions can also change the targetList , so it ' s passed to
* them by reference .
*/
qry - > sortClause = transformSortClause ( pstate ,
sstmt - > sortClause ,
& qry - > targetList ,
EXPR_KIND_ORDER_BY ,
false /* allow SQL92 rules */ ) ;
qry - > groupClause = transformGroupClause ( pstate ,
sstmt - > groupClause ,
& qry - > groupingSets ,
& qry - > targetList ,
qry - > sortClause ,
EXPR_KIND_GROUP_BY ,
false /* allow SQL92 rules */ ) ;
2021-01-22 22:26:22 +01:00
if ( sstmt - > distinctClause = = NIL )
{
qry - > distinctClause = NIL ;
qry - > hasDistinctOn = false ;
}
else if ( linitial ( sstmt - > distinctClause ) = = NULL )
{
/* We had SELECT DISTINCT */
qry - > distinctClause = transformDistinctClause ( pstate ,
& qry - > targetList ,
qry - > sortClause ,
false ) ;
qry - > hasDistinctOn = false ;
}
else
{
/* We had SELECT DISTINCT ON */
qry - > distinctClause = transformDistinctOnClause ( pstate ,
sstmt - > distinctClause ,
& qry - > targetList ,
qry - > sortClause ) ;
qry - > hasDistinctOn = true ;
}
2021-01-04 17:52:00 +01:00
/* transform LIMIT */
qry - > limitOffset = transformLimitClause ( pstate , sstmt - > limitOffset ,
EXPR_KIND_OFFSET , " OFFSET " ,
sstmt - > limitOption ) ;
qry - > limitCount = transformLimitClause ( pstate , sstmt - > limitCount ,
EXPR_KIND_LIMIT , " LIMIT " ,
sstmt - > limitOption ) ;
qry - > limitOption = sstmt - > limitOption ;
/* transform window clauses after we have seen all window functions */
qry - > windowClause = transformWindowDefinitions ( pstate ,
pstate - > p_windowdefs ,
& qry - > targetList ) ;
qry - > rtable = pstate - > p_rtable ;
qry - > jointree = makeFromExpr ( pstate - > p_joinlist , qual ) ;
qry - > hasSubLinks = pstate - > p_hasSubLinks ;
qry - > hasWindowFuncs = pstate - > p_hasWindowFuncs ;
qry - > hasTargetSRFs = pstate - > p_hasTargetSRFs ;
qry - > hasAggs = pstate - > p_hasAggs ;
foreach ( l , sstmt - > lockingClause )
{
transformLockingClause ( pstate , qry ,
( LockingClause * ) lfirst ( l ) , false ) ;
}
assign_query_collations ( pstate , qry ) ;
/* this must be done after collations, for reliable comparison of exprs */
if ( pstate - > p_hasAggs | | qry - > groupClause | | qry - > groupingSets | | qry - > havingQual )
parseCheckAggregates ( pstate , qry ) ;
return qry ;
}
2007-04-28 00:05:49 +02:00
/*
* transformDeclareCursorStmt -
* transform a DECLARE CURSOR Statement
*
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
* DECLARE CURSOR is like other utility statements in that we emit it as a
* CMD_UTILITY Query node ; however , we must first transform the contained
* query . We used to postpone that until execution , but it ' s really necessary
* to do it during the normal parse analysis phase to ensure that side effects
* of parser hooks happen at the expected time .
2007-04-28 00:05:49 +02:00
*/
static Query *
transformDeclareCursorStmt ( ParseState * pstate , DeclareCursorStmt * stmt )
{
Query * result ;
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
Query * query ;
2007-04-28 00:05:49 +02:00
if ( ( stmt - > options & CURSOR_OPT_SCROLL ) & &
( stmt - > options & CURSOR_OPT_NO_SCROLL ) )
ereport ( ERROR ,
( errcode ( ERRCODE_INVALID_CURSOR_DEFINITION ) ,
2021-04-07 07:49:27 +02:00
/* translator: %s is a SQL keyword */
errmsg ( " cannot specify both %s and %s " ,
" SCROLL " , " NO SCROLL " ) ) ) ;
if ( ( stmt - > options & CURSOR_OPT_ASENSITIVE ) & &
( stmt - > options & CURSOR_OPT_INSENSITIVE ) )
ereport ( ERROR ,
( errcode ( ERRCODE_INVALID_CURSOR_DEFINITION ) ,
/* translator: %s is a SQL keyword */
errmsg ( " cannot specify both %s and %s " ,
" ASENSITIVE " , " INSENSITIVE " ) ) ) ;
2007-04-28 00:05:49 +02:00
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
/* Transform contained query, not allowing SELECT INTO */
query = transformStmt ( pstate , stmt - > query ) ;
stmt - > query = ( Node * ) query ;
2007-04-28 00:05:49 +02:00
2008-09-01 22:42:46 +02:00
/* Grammar should not have allowed anything but SELECT */
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
if ( ! IsA ( query , Query ) | |
query - > commandType ! = CMD_SELECT )
2008-09-01 22:42:46 +02:00
elog ( ERROR , " unexpected non-SELECT command in DECLARE CURSOR " ) ;
2007-04-28 00:05:49 +02:00
2011-02-26 00:56:23 +01:00
/*
* We also disallow data - modifying WITH in a cursor . ( This could be
* allowed , but the semantics of when the updates occur might be
* surprising . )
*/
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
if ( query - > hasModifyingCTE )
2011-02-26 00:56:23 +01:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
errmsg ( " DECLARE CURSOR must not contain data-modifying statements in WITH " ) ) ) ;
2007-06-11 03:16:30 +02:00
/* FOR UPDATE and WITH HOLD are not compatible */
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
if ( query - > rowMarks ! = NIL & & ( stmt - > options & CURSOR_OPT_HOLD ) )
2007-04-28 00:05:49 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " DECLARE CURSOR WITH HOLD ... %s is not supported " ,
LCS_asString ( ( ( RowMarkClause * )
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
linitial ( query - > rowMarks ) ) - > strength ) ) ,
2007-06-11 03:16:30 +02:00
errdetail ( " Holdable cursors must be READ ONLY. " ) ) ) ;
2007-04-28 00:05:49 +02:00
2007-10-25 01:27:08 +02:00
/* FOR UPDATE and SCROLL are not compatible */
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
if ( query - > rowMarks ! = NIL & & ( stmt - > options & CURSOR_OPT_SCROLL ) )
2007-10-25 01:27:08 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " DECLARE SCROLL CURSOR ... %s is not supported " ,
LCS_asString ( ( ( RowMarkClause * )
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
linitial ( query - > rowMarks ) ) - > strength ) ) ,
2007-10-25 01:27:08 +02:00
errdetail ( " Scrollable cursors must be READ ONLY. " ) ) ) ;
/* FOR UPDATE and INSENSITIVE are not compatible */
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
if ( query - > rowMarks ! = NIL & & ( stmt - > options & CURSOR_OPT_INSENSITIVE ) )
2007-10-25 01:27:08 +02:00
ereport ( ERROR ,
2021-04-07 07:49:27 +02:00
( errcode ( ERRCODE_INVALID_CURSOR_DEFINITION ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
2021-04-07 07:49:27 +02:00
errmsg ( " DECLARE INSENSITIVE CURSOR ... %s is not valid " ,
2013-07-23 20:03:09 +02:00
LCS_asString ( ( ( RowMarkClause * )
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
linitial ( query - > rowMarks ) ) - > strength ) ) ,
2007-10-25 01:27:08 +02:00
errdetail ( " Insensitive cursors must be READ ONLY. " ) ) ) ;
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
/* represent the command as a utility Query */
result = makeNode ( Query ) ;
result - > commandType = CMD_UTILITY ;
2007-04-28 00:05:49 +02:00
result - > utilityStmt = ( Node * ) stmt ;
return result ;
}
/*
* transformExplainStmt -
* transform an EXPLAIN Statement
*
2010-01-15 23:36:35 +01:00
* EXPLAIN is like other utility statements in that we emit it as a
* CMD_UTILITY Query node ; however , we must first transform the contained
* query . We used to postpone that until execution , but it ' s really necessary
* to do it during the normal parse analysis phase to ensure that side effects
* of parser hooks happen at the expected time .
2007-04-28 00:05:49 +02:00
*/
static Query *
transformExplainStmt ( ParseState * pstate , ExplainStmt * stmt )
{
Query * result ;
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
/* transform contained query, allowing SELECT INTO */
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
stmt - > query = ( Node * ) transformOptionalSelectInto ( pstate , stmt - > query ) ;
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
/* represent the command as a utility Query */
result = makeNode ( Query ) ;
result - > commandType = CMD_UTILITY ;
result - > utilityStmt = ( Node * ) stmt ;
return result ;
}
/*
* transformCreateTableAsStmt -
2013-03-04 01:23:31 +01:00
* transform a CREATE TABLE AS , SELECT . . . INTO , or CREATE MATERIALIZED VIEW
* Statement
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
*
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
* As with DECLARE CURSOR and EXPLAIN , transform the contained statement now .
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
*/
static Query *
transformCreateTableAsStmt ( ParseState * pstate , CreateTableAsStmt * stmt )
{
Query * result ;
Clean up the mess around EXPLAIN and materialized views.
Revert the matview-related changes in explain.c's API, as per recent
complaint from Robert Haas. The reason for these appears to have been
principally some ill-considered choices around having intorel_startup do
what ought to be parse-time checking, plus a poor arrangement for passing
it the view parsetree it needs to store into pg_rewrite when creating a
materialized view. Do the latter by having parse analysis stick a copy
into the IntoClause, instead of doing it at runtime. (On the whole,
I seriously question the choice to represent CREATE MATERIALIZED VIEW as a
variant of SELECT INTO/CREATE TABLE AS, because that means injecting even
more complexity into what was already a horrid legacy kluge. However,
I didn't go so far as to rethink that choice ... yet.)
I also moved several error checks into matview parse analysis, and
made the check for external Params in a matview more accurate.
In passing, clean things up a bit more around interpretOidsOption(),
and fix things so that we can use that to force no-oids for views,
sequences, etc, thereby eliminating the need to cons up "oids = false"
options when creating them.
catversion bump due to change in IntoClause. (I wonder though if we
really need readfuncs/outfuncs support for IntoClause anymore.)
2013-04-13 01:25:20 +02:00
Query * query ;
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
/* transform contained query, not allowing SELECT INTO */
Clean up the mess around EXPLAIN and materialized views.
Revert the matview-related changes in explain.c's API, as per recent
complaint from Robert Haas. The reason for these appears to have been
principally some ill-considered choices around having intorel_startup do
what ought to be parse-time checking, plus a poor arrangement for passing
it the view parsetree it needs to store into pg_rewrite when creating a
materialized view. Do the latter by having parse analysis stick a copy
into the IntoClause, instead of doing it at runtime. (On the whole,
I seriously question the choice to represent CREATE MATERIALIZED VIEW as a
variant of SELECT INTO/CREATE TABLE AS, because that means injecting even
more complexity into what was already a horrid legacy kluge. However,
I didn't go so far as to rethink that choice ... yet.)
I also moved several error checks into matview parse analysis, and
made the check for external Params in a matview more accurate.
In passing, clean things up a bit more around interpretOidsOption(),
and fix things so that we can use that to force no-oids for views,
sequences, etc, thereby eliminating the need to cons up "oids = false"
options when creating them.
catversion bump due to change in IntoClause. (I wonder though if we
really need readfuncs/outfuncs support for IntoClause anymore.)
2013-04-13 01:25:20 +02:00
query = transformStmt ( pstate , stmt - > query ) ;
stmt - > query = ( Node * ) query ;
/* additional work needed for CREATE MATERIALIZED VIEW */
2020-07-11 06:32:28 +02:00
if ( stmt - > objtype = = OBJECT_MATVIEW )
2013-03-04 01:23:31 +01:00
{
Clean up the mess around EXPLAIN and materialized views.
Revert the matview-related changes in explain.c's API, as per recent
complaint from Robert Haas. The reason for these appears to have been
principally some ill-considered choices around having intorel_startup do
what ought to be parse-time checking, plus a poor arrangement for passing
it the view parsetree it needs to store into pg_rewrite when creating a
materialized view. Do the latter by having parse analysis stick a copy
into the IntoClause, instead of doing it at runtime. (On the whole,
I seriously question the choice to represent CREATE MATERIALIZED VIEW as a
variant of SELECT INTO/CREATE TABLE AS, because that means injecting even
more complexity into what was already a horrid legacy kluge. However,
I didn't go so far as to rethink that choice ... yet.)
I also moved several error checks into matview parse analysis, and
made the check for external Params in a matview more accurate.
In passing, clean things up a bit more around interpretOidsOption(),
and fix things so that we can use that to force no-oids for views,
sequences, etc, thereby eliminating the need to cons up "oids = false"
options when creating them.
catversion bump due to change in IntoClause. (I wonder though if we
really need readfuncs/outfuncs support for IntoClause anymore.)
2013-04-13 01:25:20 +02:00
/*
* Prohibit a data - modifying CTE in the query used to create a
* materialized view . It ' s not sufficiently clear what the user would
* want to happen if the MV is refreshed or incrementally maintained .
*/
if ( query - > hasModifyingCTE )
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
errmsg ( " materialized views must not use data-modifying statements in WITH " ) ) ) ;
2013-03-04 01:23:31 +01:00
Clean up the mess around EXPLAIN and materialized views.
Revert the matview-related changes in explain.c's API, as per recent
complaint from Robert Haas. The reason for these appears to have been
principally some ill-considered choices around having intorel_startup do
what ought to be parse-time checking, plus a poor arrangement for passing
it the view parsetree it needs to store into pg_rewrite when creating a
materialized view. Do the latter by having parse analysis stick a copy
into the IntoClause, instead of doing it at runtime. (On the whole,
I seriously question the choice to represent CREATE MATERIALIZED VIEW as a
variant of SELECT INTO/CREATE TABLE AS, because that means injecting even
more complexity into what was already a horrid legacy kluge. However,
I didn't go so far as to rethink that choice ... yet.)
I also moved several error checks into matview parse analysis, and
made the check for external Params in a matview more accurate.
In passing, clean things up a bit more around interpretOidsOption(),
and fix things so that we can use that to force no-oids for views,
sequences, etc, thereby eliminating the need to cons up "oids = false"
options when creating them.
catversion bump due to change in IntoClause. (I wonder though if we
really need readfuncs/outfuncs support for IntoClause anymore.)
2013-04-13 01:25:20 +02:00
/*
* Check whether any temporary database objects are used in the
* creation query . It would be hard to refresh data or incrementally
* maintain it if a source disappeared .
*/
if ( isQueryUsingTempRelation ( query ) )
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
errmsg ( " materialized views must not use temporary tables or views " ) ) ) ;
/*
* A materialized view would either need to save parameters for use in
* maintaining / loading the data or prohibit them entirely . The latter
* seems safer and more sane .
*/
if ( query_contains_extern_params ( query ) )
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
errmsg ( " materialized views may not be defined using bound parameters " ) ) ) ;
2013-05-06 17:57:05 +02:00
/*
* For now , we disallow unlogged materialized views , because it seems
* like a bad idea for them to just go to empty after a crash . ( If we
* could mark them as unpopulated , that would be better , but that
* requires catalog changes which crash recovery can ' t presently
* handle . )
*/
if ( stmt - > into - > rel - > relpersistence = = RELPERSISTENCE_UNLOGGED )
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2019-05-17 00:50:56 +02:00
errmsg ( " materialized views cannot be unlogged " ) ) ) ;
2013-05-06 17:57:05 +02:00
Clean up the mess around EXPLAIN and materialized views.
Revert the matview-related changes in explain.c's API, as per recent
complaint from Robert Haas. The reason for these appears to have been
principally some ill-considered choices around having intorel_startup do
what ought to be parse-time checking, plus a poor arrangement for passing
it the view parsetree it needs to store into pg_rewrite when creating a
materialized view. Do the latter by having parse analysis stick a copy
into the IntoClause, instead of doing it at runtime. (On the whole,
I seriously question the choice to represent CREATE MATERIALIZED VIEW as a
variant of SELECT INTO/CREATE TABLE AS, because that means injecting even
more complexity into what was already a horrid legacy kluge. However,
I didn't go so far as to rethink that choice ... yet.)
I also moved several error checks into matview parse analysis, and
made the check for external Params in a matview more accurate.
In passing, clean things up a bit more around interpretOidsOption(),
and fix things so that we can use that to force no-oids for views,
sequences, etc, thereby eliminating the need to cons up "oids = false"
options when creating them.
catversion bump due to change in IntoClause. (I wonder though if we
really need readfuncs/outfuncs support for IntoClause anymore.)
2013-04-13 01:25:20 +02:00
/*
* At runtime , we ' ll need a copy of the parsed - but - not - rewritten Query
* for purposes of creating the view ' s ON SELECT rule . We stash that
* in the IntoClause because that ' s where intorel_startup ( ) can
* conveniently get it from .
*/
2017-03-09 21:18:59 +01:00
stmt - > into - > viewQuery = ( Node * ) copyObject ( query ) ;
Clean up the mess around EXPLAIN and materialized views.
Revert the matview-related changes in explain.c's API, as per recent
complaint from Robert Haas. The reason for these appears to have been
principally some ill-considered choices around having intorel_startup do
what ought to be parse-time checking, plus a poor arrangement for passing
it the view parsetree it needs to store into pg_rewrite when creating a
materialized view. Do the latter by having parse analysis stick a copy
into the IntoClause, instead of doing it at runtime. (On the whole,
I seriously question the choice to represent CREATE MATERIALIZED VIEW as a
variant of SELECT INTO/CREATE TABLE AS, because that means injecting even
more complexity into what was already a horrid legacy kluge. However,
I didn't go so far as to rethink that choice ... yet.)
I also moved several error checks into matview parse analysis, and
made the check for external Params in a matview more accurate.
In passing, clean things up a bit more around interpretOidsOption(),
and fix things so that we can use that to force no-oids for views,
sequences, etc, thereby eliminating the need to cons up "oids = false"
options when creating them.
catversion bump due to change in IntoClause. (I wonder though if we
really need readfuncs/outfuncs support for IntoClause anymore.)
2013-04-13 01:25:20 +02:00
}
2007-04-28 00:05:49 +02:00
2010-01-15 23:36:35 +01:00
/* represent the command as a utility Query */
2007-04-28 00:05:49 +02:00
result = makeNode ( Query ) ;
result - > commandType = CMD_UTILITY ;
result - > utilityStmt = ( Node * ) stmt ;
return result ;
}
2018-02-21 00:03:31 +01:00
/*
* transform a CallStmt
*/
static Query *
transformCallStmt ( ParseState * pstate , CallStmt * stmt )
{
List * targs ;
ListCell * lc ;
Node * node ;
Reconsider the handling of procedure OUT parameters.
Commit 2453ea142 redefined pg_proc.proargtypes to include the types of
OUT parameters, for procedures only. While that had some advantages
for implementing the SQL-spec behavior of DROP PROCEDURE, it was pretty
disastrous from a number of other perspectives. Notably, since the
primary key of pg_proc is name + proargtypes, this made it possible to
have multiple procedures with identical names + input arguments and
differing output argument types. That would make it impossible to call
any one of the procedures by writing just NULL (or "?", or any other
data-type-free notation) for the output argument(s). The change also
seems likely to cause grave confusion for client applications that
examine pg_proc and expect the traditional definition of proargtypes.
Hence, revert the definition of proargtypes to what it was, and
undo a number of complications that had been added to support that.
To support the SQL-spec behavior of DROP PROCEDURE, when there are
no argmode markers in the command's parameter list, we perform the
lookup both ways (that is, matching against both proargtypes and
proallargtypes), succeeding if we get just one unique match.
In principle this could result in ambiguous-function failures
that would not happen when using only one of the two rules.
However, overloading of procedure names is thought to be a pretty
rare usage, so this shouldn't cause many problems in practice.
Postgres-specific code such as pg_dump can defend against any
possibility of such failures by being careful to specify argmodes
for all procedure arguments.
This also fixes a few other bugs in the area of CALL statements
with named parameters, and improves the documentation a little.
catversion bump forced because the representation of procedures
with OUT arguments changes.
Discussion: https://postgr.es/m/3742981.1621533210@sss.pgh.pa.us
2021-06-10 23:11:36 +02:00
FuncExpr * fexpr ;
HeapTuple proctup ;
Datum proargmodes ;
bool isNull ;
List * outargs = NIL ;
2018-02-21 00:03:31 +01:00
Query * result ;
Reconsider the handling of procedure OUT parameters.
Commit 2453ea142 redefined pg_proc.proargtypes to include the types of
OUT parameters, for procedures only. While that had some advantages
for implementing the SQL-spec behavior of DROP PROCEDURE, it was pretty
disastrous from a number of other perspectives. Notably, since the
primary key of pg_proc is name + proargtypes, this made it possible to
have multiple procedures with identical names + input arguments and
differing output argument types. That would make it impossible to call
any one of the procedures by writing just NULL (or "?", or any other
data-type-free notation) for the output argument(s). The change also
seems likely to cause grave confusion for client applications that
examine pg_proc and expect the traditional definition of proargtypes.
Hence, revert the definition of proargtypes to what it was, and
undo a number of complications that had been added to support that.
To support the SQL-spec behavior of DROP PROCEDURE, when there are
no argmode markers in the command's parameter list, we perform the
lookup both ways (that is, matching against both proargtypes and
proallargtypes), succeeding if we get just one unique match.
In principle this could result in ambiguous-function failures
that would not happen when using only one of the two rules.
However, overloading of procedure names is thought to be a pretty
rare usage, so this shouldn't cause many problems in practice.
Postgres-specific code such as pg_dump can defend against any
possibility of such failures by being careful to specify argmodes
for all procedure arguments.
This also fixes a few other bugs in the area of CALL statements
with named parameters, and improves the documentation a little.
catversion bump forced because the representation of procedures
with OUT arguments changes.
Discussion: https://postgr.es/m/3742981.1621533210@sss.pgh.pa.us
2021-06-10 23:11:36 +02:00
/*
* First , do standard parse analysis on the procedure call and its
* arguments , allowing us to identify the called procedure .
*/
2018-02-21 00:03:31 +01:00
targs = NIL ;
foreach ( lc , stmt - > funccall - > args )
{
targs = lappend ( targs , transformExpr ( pstate ,
( Node * ) lfirst ( lc ) ,
EXPR_KIND_CALL_ARGUMENT ) ) ;
}
node = ParseFuncOrColumn ( pstate ,
stmt - > funccall - > funcname ,
targs ,
pstate - > p_last_srf ,
stmt - > funccall ,
true ,
stmt - > funccall - > location ) ;
2019-02-05 15:08:53 +01:00
assign_expr_collations ( pstate , node ) ;
Reconsider the handling of procedure OUT parameters.
Commit 2453ea142 redefined pg_proc.proargtypes to include the types of
OUT parameters, for procedures only. While that had some advantages
for implementing the SQL-spec behavior of DROP PROCEDURE, it was pretty
disastrous from a number of other perspectives. Notably, since the
primary key of pg_proc is name + proargtypes, this made it possible to
have multiple procedures with identical names + input arguments and
differing output argument types. That would make it impossible to call
any one of the procedures by writing just NULL (or "?", or any other
data-type-free notation) for the output argument(s). The change also
seems likely to cause grave confusion for client applications that
examine pg_proc and expect the traditional definition of proargtypes.
Hence, revert the definition of proargtypes to what it was, and
undo a number of complications that had been added to support that.
To support the SQL-spec behavior of DROP PROCEDURE, when there are
no argmode markers in the command's parameter list, we perform the
lookup both ways (that is, matching against both proargtypes and
proallargtypes), succeeding if we get just one unique match.
In principle this could result in ambiguous-function failures
that would not happen when using only one of the two rules.
However, overloading of procedure names is thought to be a pretty
rare usage, so this shouldn't cause many problems in practice.
Postgres-specific code such as pg_dump can defend against any
possibility of such failures by being careful to specify argmodes
for all procedure arguments.
This also fixes a few other bugs in the area of CALL statements
with named parameters, and improves the documentation a little.
catversion bump forced because the representation of procedures
with OUT arguments changes.
Discussion: https://postgr.es/m/3742981.1621533210@sss.pgh.pa.us
2021-06-10 23:11:36 +02:00
fexpr = castNode ( FuncExpr , node ) ;
proctup = SearchSysCache1 ( PROCOID , ObjectIdGetDatum ( fexpr - > funcid ) ) ;
if ( ! HeapTupleIsValid ( proctup ) )
elog ( ERROR , " cache lookup failed for function %u " , fexpr - > funcid ) ;
2018-02-21 00:03:31 +01:00
Reconsider the handling of procedure OUT parameters.
Commit 2453ea142 redefined pg_proc.proargtypes to include the types of
OUT parameters, for procedures only. While that had some advantages
for implementing the SQL-spec behavior of DROP PROCEDURE, it was pretty
disastrous from a number of other perspectives. Notably, since the
primary key of pg_proc is name + proargtypes, this made it possible to
have multiple procedures with identical names + input arguments and
differing output argument types. That would make it impossible to call
any one of the procedures by writing just NULL (or "?", or any other
data-type-free notation) for the output argument(s). The change also
seems likely to cause grave confusion for client applications that
examine pg_proc and expect the traditional definition of proargtypes.
Hence, revert the definition of proargtypes to what it was, and
undo a number of complications that had been added to support that.
To support the SQL-spec behavior of DROP PROCEDURE, when there are
no argmode markers in the command's parameter list, we perform the
lookup both ways (that is, matching against both proargtypes and
proallargtypes), succeeding if we get just one unique match.
In principle this could result in ambiguous-function failures
that would not happen when using only one of the two rules.
However, overloading of procedure names is thought to be a pretty
rare usage, so this shouldn't cause many problems in practice.
Postgres-specific code such as pg_dump can defend against any
possibility of such failures by being careful to specify argmodes
for all procedure arguments.
This also fixes a few other bugs in the area of CALL statements
with named parameters, and improves the documentation a little.
catversion bump forced because the representation of procedures
with OUT arguments changes.
Discussion: https://postgr.es/m/3742981.1621533210@sss.pgh.pa.us
2021-06-10 23:11:36 +02:00
/*
* Expand the argument list to deal with named - argument notation and
* default arguments . For ordinary FuncExprs this ' d be done during
* planning , but a CallStmt doesn ' t go through planning , and there seems
* no good reason not to do it here .
*/
fexpr - > args = expand_function_arguments ( fexpr - > args ,
true ,
fexpr - > funcresulttype ,
proctup ) ;
/* Fetch proargmodes; if it's null, there are no output args */
proargmodes = SysCacheGetAttr ( PROCOID , proctup ,
Anum_pg_proc_proargmodes ,
& isNull ) ;
if ( ! isNull )
{
/*
* Split the list into input arguments in fexpr - > args and output
* arguments in stmt - > outargs . INOUT arguments appear in both lists .
*/
ArrayType * arr ;
int numargs ;
char * argmodes ;
List * inargs ;
int i ;
arr = DatumGetArrayTypeP ( proargmodes ) ; /* ensure not toasted */
numargs = list_length ( fexpr - > args ) ;
if ( ARR_NDIM ( arr ) ! = 1 | |
ARR_DIMS ( arr ) [ 0 ] ! = numargs | |
ARR_HASNULL ( arr ) | |
ARR_ELEMTYPE ( arr ) ! = CHAROID )
elog ( ERROR , " proargmodes is not a 1-D char array of length %d or it contains nulls " ,
numargs ) ;
argmodes = ( char * ) ARR_DATA_PTR ( arr ) ;
inargs = NIL ;
i = 0 ;
foreach ( lc , fexpr - > args )
{
Node * n = lfirst ( lc ) ;
switch ( argmodes [ i ] )
{
case PROARGMODE_IN :
case PROARGMODE_VARIADIC :
inargs = lappend ( inargs , n ) ;
break ;
case PROARGMODE_OUT :
outargs = lappend ( outargs , n ) ;
break ;
case PROARGMODE_INOUT :
inargs = lappend ( inargs , n ) ;
outargs = lappend ( outargs , copyObject ( n ) ) ;
break ;
default :
/* note we don't support PROARGMODE_TABLE */
elog ( ERROR , " invalid argmode %c for procedure " ,
argmodes [ i ] ) ;
break ;
}
i + + ;
}
fexpr - > args = inargs ;
}
stmt - > funcexpr = fexpr ;
stmt - > outargs = outargs ;
ReleaseSysCache ( proctup ) ;
/* represent the command as a utility Query */
2018-02-21 00:03:31 +01:00
result = makeNode ( Query ) ;
result - > commandType = CMD_UTILITY ;
result - > utilityStmt = ( Node * ) stmt ;
return result ;
}
2007-04-28 00:05:49 +02:00
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
/*
* Produce a string representation of a LockClauseStrength value .
* This should only be applied to valid values ( not LCS_NONE ) .
*/
const char *
2013-07-23 20:03:09 +02:00
LCS_asString ( LockClauseStrength strength )
{
switch ( strength )
{
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
case LCS_NONE :
Assert ( false ) ;
break ;
2013-07-23 20:03:09 +02:00
case LCS_FORKEYSHARE :
return " FOR KEY SHARE " ;
case LCS_FORSHARE :
return " FOR SHARE " ;
case LCS_FORNOKEYUPDATE :
return " FOR NO KEY UPDATE " ;
case LCS_FORUPDATE :
return " FOR UPDATE " ;
}
return " FOR some " ; /* shouldn't happen */
}
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
/*
2013-03-10 19:14:53 +01:00
* Check for features that are not supported with FOR [ KEY ] UPDATE / SHARE .
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
*
* exported so planner can check again after rewriting , query pullup , etc
*/
1999-01-25 13:01:19 +01:00
void
2013-08-02 18:49:03 +02:00
CheckSelectLocking ( Query * qry , LockClauseStrength strength )
1999-01-25 13:01:19 +01:00
{
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
Assert ( strength ! = LCS_NONE ) ; /* else caller error */
2000-10-05 21:11:39 +02:00
if ( qry - > setOperations )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s is not allowed with UNION/INTERSECT/EXCEPT " ,
2013-08-02 18:49:03 +02:00
LCS_asString ( strength ) ) ) ) ;
2000-01-27 19:11:50 +01:00
if ( qry - > distinctClause ! = NIL )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s is not allowed with DISTINCT clause " ,
2013-08-02 18:49:03 +02:00
LCS_asString ( strength ) ) ) ) ;
2021-06-01 17:12:56 +02:00
if ( qry - > groupClause ! = NIL | | qry - > groupingSets ! = NIL )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s is not allowed with GROUP BY clause " ,
2013-08-02 18:49:03 +02:00
LCS_asString ( strength ) ) ) ) ;
2005-03-11 00:21:26 +01:00
if ( qry - > havingQual ! = NULL )
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s is not allowed with HAVING clause " ,
2013-08-02 18:49:03 +02:00
LCS_asString ( strength ) ) ) ) ;
1999-01-25 13:01:19 +01:00
if ( qry - > hasAggs )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s is not allowed with aggregate functions " ,
2013-08-02 18:49:03 +02:00
LCS_asString ( strength ) ) ) ) ;
2008-12-28 19:54:01 +01:00
if ( qry - > hasWindowFuncs )
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s is not allowed with window functions " ,
2013-08-02 18:49:03 +02:00
LCS_asString ( strength ) ) ) ) ;
2016-09-13 19:54:24 +02:00
if ( qry - > hasTargetSRFs )
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s is not allowed with set-returning functions in the target list " ,
2013-08-02 18:49:03 +02:00
LCS_asString ( strength ) ) ) ) ;
1999-01-25 13:01:19 +01:00
}
2003-11-05 23:00:46 +01:00
/*
Improve concurrency of foreign key locking
This patch introduces two additional lock modes for tuples: "SELECT FOR
KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each
other, in contrast with already existing "SELECT FOR SHARE" and "SELECT
FOR UPDATE". UPDATE commands that do not modify the values stored in
the columns that are part of the key of the tuple now grab a SELECT FOR
NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently
with tuple locks of the FOR KEY SHARE variety.
Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this
means the concurrency improvement applies to them, which is the whole
point of this patch.
The added tuple lock semantics require some rejiggering of the multixact
module, so that the locking level that each transaction is holding can
be stored alongside its Xid. Also, multixacts now need to persist
across server restarts and crashes, because they can now represent not
only tuple locks, but also tuple updates. This means we need more
careful tracking of lifetime of pg_multixact SLRU files; since they now
persist longer, we require more infrastructure to figure out when they
can be removed. pg_upgrade also needs to be careful to copy
pg_multixact files over from the old server to the new, or at least part
of multixact.c state, depending on the versions of the old and new
servers.
Tuple time qualification rules (HeapTupleSatisfies routines) need to be
careful not to consider tuples with the "is multi" infomask bit set as
being only locked; they might need to look up MultiXact values (i.e.
possibly do pg_multixact I/O) to find out the Xid that updated a tuple,
whereas they previously were assured to only use information readily
available from the tuple header. This is considered acceptable, because
the extra I/O would involve cases that would previously cause some
commands to block waiting for concurrent transactions to finish.
Another important change is the fact that locking tuples that have
previously been updated causes the future versions to be marked as
locked, too; this is essential for correctness of foreign key checks.
This causes additional WAL-logging, also (there was previously a single
WAL record for a locked tuple; now there are as many as updated copies
of the tuple there exist.)
With all this in place, contention related to tuples being checked by
foreign key rules should be much reduced.
As a bonus, the old behavior that a subtransaction grabbing a stronger
tuple lock than the parent (sub)transaction held on a given tuple and
later aborting caused the weaker lock to be lost, has been fixed.
Many new spec files were added for isolation tester framework, to ensure
overall behavior is sane. There's probably room for several more tests.
There were several reviewers of this patch; in particular, Noah Misch
and Andres Freund spent considerable time in it. Original idea for the
patch came from Simon Riggs, after a problem report by Joel Jacobson.
Most code is from me, with contributions from Marti Raudsepp, Alexander
Shulgin, Noah Misch and Andres Freund.
This patch was discussed in several pgsql-hackers threads; the most
important start at the following message-ids:
AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com
1290721684-sup-3951@alvh.no-ip.org
1294953201-sup-2099@alvh.no-ip.org
1320343602-sup-2290@alvh.no-ip.org
1339690386-sup-8927@alvh.no-ip.org
4FE5FF020200002500048A3D@gw.wicourts.gov
4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
* Transform a FOR [ KEY ] UPDATE / SHARE clause
2006-04-30 20:30:40 +02:00
*
* This basically involves replacing names by integer relids .
2003-11-05 23:00:46 +01:00
*
2005-04-28 23:47:18 +02:00
* NB : if you need to change this , see also markQueryForLocking ( )
2009-10-27 18:11:18 +01:00
* in rewriteHandler . c , and isLockedRefname ( ) in parse_relation . c .
2003-11-05 23:00:46 +01:00
*/
1999-01-21 17:08:55 +01:00
static void
2009-10-28 15:55:47 +01:00
transformLockingClause ( ParseState * pstate , Query * qry , LockingClause * lc ,
bool pushedDown )
1999-01-21 17:08:55 +01:00
{
2005-08-01 22:31:16 +02:00
List * lockedRels = lc - > lockedRels ;
2004-05-26 06:41:50 +02:00
ListCell * l ;
ListCell * rt ;
1999-01-21 17:08:55 +01:00
Index i ;
2005-08-01 22:31:16 +02:00
LockingClause * allrels ;
1999-01-21 17:08:55 +01:00
2013-08-02 18:49:03 +02:00
CheckSelectLocking ( qry , lc - > strength ) ;
2005-08-01 22:31:16 +02:00
/* make a clause we can pass down to subqueries to select all rels */
allrels = makeNode ( LockingClause ) ;
allrels - > lockedRels = NIL ; /* indicates all rels */
Improve concurrency of foreign key locking
This patch introduces two additional lock modes for tuples: "SELECT FOR
KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each
other, in contrast with already existing "SELECT FOR SHARE" and "SELECT
FOR UPDATE". UPDATE commands that do not modify the values stored in
the columns that are part of the key of the tuple now grab a SELECT FOR
NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently
with tuple locks of the FOR KEY SHARE variety.
Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this
means the concurrency improvement applies to them, which is the whole
point of this patch.
The added tuple lock semantics require some rejiggering of the multixact
module, so that the locking level that each transaction is holding can
be stored alongside its Xid. Also, multixacts now need to persist
across server restarts and crashes, because they can now represent not
only tuple locks, but also tuple updates. This means we need more
careful tracking of lifetime of pg_multixact SLRU files; since they now
persist longer, we require more infrastructure to figure out when they
can be removed. pg_upgrade also needs to be careful to copy
pg_multixact files over from the old server to the new, or at least part
of multixact.c state, depending on the versions of the old and new
servers.
Tuple time qualification rules (HeapTupleSatisfies routines) need to be
careful not to consider tuples with the "is multi" infomask bit set as
being only locked; they might need to look up MultiXact values (i.e.
possibly do pg_multixact I/O) to find out the Xid that updated a tuple,
whereas they previously were assured to only use information readily
available from the tuple header. This is considered acceptable, because
the extra I/O would involve cases that would previously cause some
commands to block waiting for concurrent transactions to finish.
Another important change is the fact that locking tuples that have
previously been updated causes the future versions to be marked as
locked, too; this is essential for correctness of foreign key checks.
This causes additional WAL-logging, also (there was previously a single
WAL record for a locked tuple; now there are as many as updated copies
of the tuple there exist.)
With all this in place, contention related to tuples being checked by
foreign key rules should be much reduced.
As a bonus, the old behavior that a subtransaction grabbing a stronger
tuple lock than the parent (sub)transaction held on a given tuple and
later aborting caused the weaker lock to be lost, has been fixed.
Many new spec files were added for isolation tester framework, to ensure
overall behavior is sane. There's probably room for several more tests.
There were several reviewers of this patch; in particular, Noah Misch
and Andres Freund spent considerable time in it. Original idea for the
patch came from Simon Riggs, after a problem report by Joel Jacobson.
Most code is from me, with contributions from Marti Raudsepp, Alexander
Shulgin, Noah Misch and Andres Freund.
This patch was discussed in several pgsql-hackers threads; the most
important start at the following message-ids:
AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com
1290721684-sup-3951@alvh.no-ip.org
1294953201-sup-2099@alvh.no-ip.org
1320343602-sup-2290@alvh.no-ip.org
1339690386-sup-8927@alvh.no-ip.org
4FE5FF020200002500048A3D@gw.wicourts.gov
4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
allrels - > strength = lc - > strength ;
2014-10-07 22:23:34 +02:00
allrels - > waitPolicy = lc - > waitPolicy ;
1999-01-25 13:01:19 +01:00
2005-08-01 22:31:16 +02:00
if ( lockedRels = = NIL )
1999-01-21 17:08:55 +01:00
{
2021-08-19 18:12:35 +02:00
/*
* Lock all regular tables used in query and its subqueries . We
* examine inFromCl to exclude auto - added RTEs , particularly NEW / OLD
* in rules . This is a bit of an abuse of a mostly - obsolete flag , but
* it ' s convenient . We can ' t rely on the namespace mechanism that has
* largely replaced inFromCl , since for example we need to lock
* base - relation RTEs even if they are masked by upper joins .
*/
2000-09-29 20:21:41 +02:00
i = 0 ;
foreach ( rt , qry - > rtable )
1999-01-21 17:08:55 +01:00
{
2000-09-29 20:21:41 +02:00
RangeTblEntry * rte = ( RangeTblEntry * ) lfirst ( rt ) ;
+ + i ;
2021-08-19 18:12:35 +02:00
if ( ! rte - > inFromCl )
continue ;
2003-11-05 23:00:46 +01:00
switch ( rte - > rtekind )
2000-09-29 20:21:41 +02:00
{
2003-11-05 23:00:46 +01:00
case RTE_RELATION :
2014-10-07 22:23:34 +02:00
applyLockingClause ( qry , i , lc - > strength , lc - > waitPolicy ,
pushedDown ) ;
2011-02-23 01:23:23 +01:00
rte - > requiredPerms | = ACL_SELECT_FOR_UPDATE ;
2003-11-05 23:00:46 +01:00
break ;
case RTE_SUBQUERY :
2014-10-07 22:23:34 +02:00
applyLockingClause ( qry , i , lc - > strength , lc - > waitPolicy ,
pushedDown ) ;
2010-02-26 03:01:40 +01:00
2003-11-05 23:00:46 +01:00
/*
2013-03-10 19:14:53 +01:00
* FOR UPDATE / SHARE of subquery is propagated to all of
2009-10-28 15:55:47 +01:00
* subquery ' s rels , too . We could do this later ( based on
* the marking of the subquery RTE ) but it is convenient
* to have local knowledge in each query level about which
* rels need to be opened with RowShareLock .
2003-11-05 23:00:46 +01:00
*/
2009-10-28 15:55:47 +01:00
transformLockingClause ( pstate , rte - > subquery ,
allrels , true ) ;
2003-11-05 23:00:46 +01:00
break ;
default :
2009-10-27 18:11:18 +01:00
/* ignore JOIN, SPECIAL, FUNCTION, VALUES, CTE RTEs */
2003-11-05 23:00:46 +01:00
break ;
2000-09-29 20:21:41 +02:00
}
1999-01-21 17:08:55 +01:00
}
}
2000-09-29 20:21:41 +02:00
else
1999-01-21 17:08:55 +01:00
{
2021-08-19 18:12:35 +02:00
/*
* Lock just the named tables . As above , we allow locking any base
* relation regardless of alias - visibility rules , so we need to
* examine inFromCl to exclude OLD / NEW .
*/
2005-04-28 23:47:18 +02:00
foreach ( l , lockedRels )
1999-01-21 17:08:55 +01:00
{
2008-09-01 22:42:46 +02:00
RangeVar * thisrel = ( RangeVar * ) lfirst ( l ) ;
/* For simplicity we insist on unqualified alias names here */
if ( thisrel - > catalogname | | thisrel - > schemaname )
ereport ( ERROR ,
( errcode ( ERRCODE_SYNTAX_ERROR ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s must specify unqualified relation names " ,
LCS_asString ( lc - > strength ) ) ,
2008-09-01 22:42:46 +02:00
parser_errposition ( pstate , thisrel - > location ) ) ) ;
2000-09-29 20:21:41 +02:00
i = 0 ;
foreach ( rt , qry - > rtable )
1999-01-21 17:08:55 +01:00
{
2000-09-29 20:21:41 +02:00
RangeTblEntry * rte = ( RangeTblEntry * ) lfirst ( rt ) ;
2000-01-16 09:21:59 +01:00
2000-09-29 20:21:41 +02:00
+ + i ;
2021-08-19 18:12:35 +02:00
if ( ! rte - > inFromCl )
continue ;
2008-09-01 22:42:46 +02:00
if ( strcmp ( rte - > eref - > aliasname , thisrel - > relname ) = = 0 )
1999-01-21 17:08:55 +01:00
{
2003-11-05 23:00:46 +01:00
switch ( rte - > rtekind )
2000-09-29 20:21:41 +02:00
{
2003-11-05 23:00:46 +01:00
case RTE_RELATION :
2014-10-07 22:23:34 +02:00
applyLockingClause ( qry , i , lc - > strength ,
lc - > waitPolicy , pushedDown ) ;
2004-01-15 00:01:55 +01:00
rte - > requiredPerms | = ACL_SELECT_FOR_UPDATE ;
2003-11-05 23:00:46 +01:00
break ;
case RTE_SUBQUERY :
2014-10-07 22:23:34 +02:00
applyLockingClause ( qry , i , lc - > strength ,
lc - > waitPolicy , pushedDown ) ;
2009-10-28 15:55:47 +01:00
/* see comment above */
transformLockingClause ( pstate , rte - > subquery ,
allrels , true ) ;
2003-11-05 23:00:46 +01:00
break ;
case RTE_JOIN :
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s cannot be applied to a join " ,
LCS_asString ( lc - > strength ) ) ,
2008-09-01 22:42:46 +02:00
parser_errposition ( pstate , thisrel - > location ) ) ) ;
2003-11-05 23:00:46 +01:00
break ;
case RTE_FUNCTION :
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s cannot be applied to a function " ,
LCS_asString ( lc - > strength ) ) ,
2008-09-01 22:42:46 +02:00
parser_errposition ( pstate , thisrel - > location ) ) ) ;
2003-11-05 23:00:46 +01:00
break ;
2017-03-08 16:39:37 +01:00
case RTE_TABLEFUNC :
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s cannot be applied to a table function " ,
LCS_asString ( lc - > strength ) ) ,
parser_errposition ( pstate , thisrel - > location ) ) ) ;
break ;
2006-08-02 03:59:48 +02:00
case RTE_VALUES :
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s cannot be applied to VALUES " ,
LCS_asString ( lc - > strength ) ) ,
2008-09-01 22:42:46 +02:00
parser_errposition ( pstate , thisrel - > location ) ) ) ;
2006-08-02 03:59:48 +02:00
break ;
2008-10-04 23:56:55 +02:00
case RTE_CTE :
2009-10-27 18:11:18 +01:00
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s cannot be applied to a WITH query " ,
LCS_asString ( lc - > strength ) ) ,
2017-04-01 06:17:18 +02:00
parser_errposition ( pstate , thisrel - > location ) ) ) ;
break ;
case RTE_NAMEDTUPLESTORE :
ereport ( ERROR ,
( errcode ( ERRCODE_FEATURE_NOT_SUPPORTED ) ,
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " %s cannot be applied to a named tuplestore " ,
LCS_asString ( lc - > strength ) ) ,
2009-10-27 18:11:18 +01:00
parser_errposition ( pstate , thisrel - > location ) ) ) ;
2008-10-04 23:56:55 +02:00
break ;
In the planner, replace an empty FROM clause with a dummy RTE.
The fact that "SELECT expression" has no base relations has long been a
thorn in the side of the planner. It makes it hard to flatten a sub-query
that looks like that, or is a trivial VALUES() item, because the planner
generally uses relid sets to identify sub-relations, and such a sub-query
would have an empty relid set if we flattened it. prepjointree.c contains
some baroque logic that works around this in certain special cases --- but
there is a much better answer. We can replace an empty FROM clause with a
dummy RTE that acts like a table of one row and no columns, and then there
are no such corner cases to worry about. Instead we need some logic to
get rid of useless dummy RTEs, but that's simpler and covers more cases
than what was there before.
For really trivial cases, where the query is just "SELECT expression" and
nothing else, there's a hazard that adding the extra RTE makes for a
noticeable slowdown; even though it's not much processing, there's not
that much for the planner to do overall. However testing says that the
penalty is very small, close to the noise level. In more complex queries,
this is able to find optimizations that we could not find before.
The new RTE type is called RTE_RESULT, since the "scan" plan type it
gives rise to is a Result node (the same plan we produced for a "SELECT
expression" query before). To avoid confusion, rename the old ResultPath
path type to GroupResultPath, reflecting that it's only used in degenerate
grouping cases where we know the query produces just one grouped row.
(It wouldn't work to unify the two cases, because there are different
rules about where the associated quals live during query_planner.)
Note: although this touches readfuncs.c, I don't think a catversion
bump is required, because the added case can't occur in stored rules,
only plans.
Patch by me, reviewed by David Rowley and Mark Dilger
Discussion: https://postgr.es/m/15944.1521127664@sss.pgh.pa.us
2019-01-28 23:54:10 +01:00
/* Shouldn't be possible to see RTE_RESULT here */
2003-11-05 23:00:46 +01:00
default :
elog ( ERROR , " unrecognized RTE type: %d " ,
( int ) rte - > rtekind ) ;
break ;
2000-09-29 20:21:41 +02:00
}
2003-11-05 23:00:46 +01:00
break ; /* out of foreach loop */
1999-01-21 17:08:55 +01:00
}
}
2004-05-26 06:41:50 +02:00
if ( rt = = NULL )
2003-07-19 22:20:53 +02:00
ereport ( ERROR ,
( errcode ( ERRCODE_UNDEFINED_TABLE ) ,
2013-07-23 20:03:09 +02:00
/*------
translator : % s is a SQL row locking clause such as FOR UPDATE */
errmsg ( " relation \" %s \" in %s clause not found in FROM clause " ,
thisrel - > relname ,
LCS_asString ( lc - > strength ) ) ,
2008-09-01 22:42:46 +02:00
parser_errposition ( pstate , thisrel - > location ) ) ) ;
1999-01-21 17:08:55 +01:00
}
}
2006-04-30 20:30:40 +02:00
}
/*
* Record locking info for a single rangetable item
*/
void
2009-10-28 15:55:47 +01:00
applyLockingClause ( Query * qry , Index rtindex ,
2014-10-07 22:23:34 +02:00
LockClauseStrength strength , LockWaitPolicy waitPolicy ,
bool pushedDown )
2006-04-30 20:30:40 +02:00
{
RowMarkClause * rc ;
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
Assert ( strength ! = LCS_NONE ) ; /* else caller error */
2009-10-28 15:55:47 +01:00
/* If it's an explicit clause, make sure hasForUpdate gets set */
if ( ! pushedDown )
qry - > hasForUpdate = true ;
2006-04-30 20:30:40 +02:00
/* Check for pre-existing entry for same rtindex */
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
if ( ( rc = get_parse_rowmark ( qry , rtindex ) ) ! = NULL )
2006-04-30 20:30:40 +02:00
{
/*
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
* If the same RTE is specified with more than one locking strength ,
* use the strongest . ( Reasonable , since you can ' t take both a shared
* and exclusive lock at the same time ; it ' ll end up being exclusive
* anyway . )
2006-04-30 20:30:40 +02:00
*
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
* Similarly , if the same RTE is specified with more than one lock
* wait policy , consider that NOWAIT wins over SKIP LOCKED , which in
* turn wins over waiting for the lock ( the default ) . This is a bit
* more debatable but raising an error doesn ' t seem helpful . ( Consider
* for instance SELECT FOR UPDATE NOWAIT from a view that internally
2014-10-07 22:23:34 +02:00
* contains a plain FOR UPDATE spec . ) Having NOWAIT win over SKIP
* LOCKED is reasonable since the former throws an error in case of
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
* coming across a locked tuple , which may be undesirable in some
* cases but it seems better than silently returning inconsistent
* results .
2009-10-28 15:55:47 +01:00
*
* And of course pushedDown becomes false if any clause is explicit .
2006-04-30 20:30:40 +02:00
*/
Improve concurrency of foreign key locking
This patch introduces two additional lock modes for tuples: "SELECT FOR
KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each
other, in contrast with already existing "SELECT FOR SHARE" and "SELECT
FOR UPDATE". UPDATE commands that do not modify the values stored in
the columns that are part of the key of the tuple now grab a SELECT FOR
NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently
with tuple locks of the FOR KEY SHARE variety.
Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this
means the concurrency improvement applies to them, which is the whole
point of this patch.
The added tuple lock semantics require some rejiggering of the multixact
module, so that the locking level that each transaction is holding can
be stored alongside its Xid. Also, multixacts now need to persist
across server restarts and crashes, because they can now represent not
only tuple locks, but also tuple updates. This means we need more
careful tracking of lifetime of pg_multixact SLRU files; since they now
persist longer, we require more infrastructure to figure out when they
can be removed. pg_upgrade also needs to be careful to copy
pg_multixact files over from the old server to the new, or at least part
of multixact.c state, depending on the versions of the old and new
servers.
Tuple time qualification rules (HeapTupleSatisfies routines) need to be
careful not to consider tuples with the "is multi" infomask bit set as
being only locked; they might need to look up MultiXact values (i.e.
possibly do pg_multixact I/O) to find out the Xid that updated a tuple,
whereas they previously were assured to only use information readily
available from the tuple header. This is considered acceptable, because
the extra I/O would involve cases that would previously cause some
commands to block waiting for concurrent transactions to finish.
Another important change is the fact that locking tuples that have
previously been updated causes the future versions to be marked as
locked, too; this is essential for correctness of foreign key checks.
This causes additional WAL-logging, also (there was previously a single
WAL record for a locked tuple; now there are as many as updated copies
of the tuple there exist.)
With all this in place, contention related to tuples being checked by
foreign key rules should be much reduced.
As a bonus, the old behavior that a subtransaction grabbing a stronger
tuple lock than the parent (sub)transaction held on a given tuple and
later aborting caused the weaker lock to be lost, has been fixed.
Many new spec files were added for isolation tester framework, to ensure
overall behavior is sane. There's probably room for several more tests.
There were several reviewers of this patch; in particular, Noah Misch
and Andres Freund spent considerable time in it. Original idea for the
patch came from Simon Riggs, after a problem report by Joel Jacobson.
Most code is from me, with contributions from Marti Raudsepp, Alexander
Shulgin, Noah Misch and Andres Freund.
This patch was discussed in several pgsql-hackers threads; the most
important start at the following message-ids:
AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com
1290721684-sup-3951@alvh.no-ip.org
1294953201-sup-2099@alvh.no-ip.org
1320343602-sup-2290@alvh.no-ip.org
1339690386-sup-8927@alvh.no-ip.org
4FE5FF020200002500048A3D@gw.wicourts.gov
4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
rc - > strength = Max ( rc - > strength , strength ) ;
2014-10-07 22:23:34 +02:00
rc - > waitPolicy = Max ( rc - > waitPolicy , waitPolicy ) ;
2009-10-28 15:55:47 +01:00
rc - > pushedDown & = pushedDown ;
2006-04-30 20:30:40 +02:00
return ;
}
1999-01-21 17:08:55 +01:00
2006-04-30 20:30:40 +02:00
/* Make a new RowMarkClause */
rc = makeNode ( RowMarkClause ) ;
rc - > rti = rtindex ;
Improve concurrency of foreign key locking
This patch introduces two additional lock modes for tuples: "SELECT FOR
KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each
other, in contrast with already existing "SELECT FOR SHARE" and "SELECT
FOR UPDATE". UPDATE commands that do not modify the values stored in
the columns that are part of the key of the tuple now grab a SELECT FOR
NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently
with tuple locks of the FOR KEY SHARE variety.
Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this
means the concurrency improvement applies to them, which is the whole
point of this patch.
The added tuple lock semantics require some rejiggering of the multixact
module, so that the locking level that each transaction is holding can
be stored alongside its Xid. Also, multixacts now need to persist
across server restarts and crashes, because they can now represent not
only tuple locks, but also tuple updates. This means we need more
careful tracking of lifetime of pg_multixact SLRU files; since they now
persist longer, we require more infrastructure to figure out when they
can be removed. pg_upgrade also needs to be careful to copy
pg_multixact files over from the old server to the new, or at least part
of multixact.c state, depending on the versions of the old and new
servers.
Tuple time qualification rules (HeapTupleSatisfies routines) need to be
careful not to consider tuples with the "is multi" infomask bit set as
being only locked; they might need to look up MultiXact values (i.e.
possibly do pg_multixact I/O) to find out the Xid that updated a tuple,
whereas they previously were assured to only use information readily
available from the tuple header. This is considered acceptable, because
the extra I/O would involve cases that would previously cause some
commands to block waiting for concurrent transactions to finish.
Another important change is the fact that locking tuples that have
previously been updated causes the future versions to be marked as
locked, too; this is essential for correctness of foreign key checks.
This causes additional WAL-logging, also (there was previously a single
WAL record for a locked tuple; now there are as many as updated copies
of the tuple there exist.)
With all this in place, contention related to tuples being checked by
foreign key rules should be much reduced.
As a bonus, the old behavior that a subtransaction grabbing a stronger
tuple lock than the parent (sub)transaction held on a given tuple and
later aborting caused the weaker lock to be lost, has been fixed.
Many new spec files were added for isolation tester framework, to ensure
overall behavior is sane. There's probably room for several more tests.
There were several reviewers of this patch; in particular, Noah Misch
and Andres Freund spent considerable time in it. Original idea for the
patch came from Simon Riggs, after a problem report by Joel Jacobson.
Most code is from me, with contributions from Marti Raudsepp, Alexander
Shulgin, Noah Misch and Andres Freund.
This patch was discussed in several pgsql-hackers threads; the most
important start at the following message-ids:
AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com
1290721684-sup-3951@alvh.no-ip.org
1294953201-sup-2099@alvh.no-ip.org
1320343602-sup-2290@alvh.no-ip.org
1339690386-sup-8927@alvh.no-ip.org
4FE5FF020200002500048A3D@gw.wicourts.gov
4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
rc - > strength = strength ;
2014-10-07 22:23:34 +02:00
rc - > waitPolicy = waitPolicy ;
2009-10-28 15:55:47 +01:00
rc - > pushedDown = pushedDown ;
2006-04-30 20:30:40 +02:00
qry - > rowMarks = lappend ( qry - > rowMarks , rc ) ;
1999-01-21 17:08:55 +01:00
}
2016-05-24 01:08:26 +02:00
/*
* Coverage testing for raw_expression_tree_walker ( ) .
*
* When enabled , we run raw_expression_tree_walker ( ) over every DML statement
* submitted to parse analysis . Without this provision , that function is only
* applied in limited cases involving CTEs , and we don ' t really want to have
* to test everything inside as well as outside a CTE .
*/
# ifdef RAW_EXPRESSION_COVERAGE_TEST
static bool
test_raw_expression_coverage ( Node * node , void * context )
{
if ( node = = NULL )
return false ;
return raw_expression_tree_walker ( node ,
test_raw_expression_coverage ,
context ) ;
}
# endif /* RAW_EXPRESSION_COVERAGE_TEST */