postgresql/src/backend/rewrite/rewriteHandler.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

4248 lines
133 KiB
C
Raw Normal View History

/*-------------------------------------------------------------------------
*
* rewriteHandler.c
* Primary module of query rewriter.
*
* Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
2010-09-20 22:08:53 +02:00
* src/backend/rewrite/rewriteHandler.c
*
* NOTES
* Some of the terms used in this file are of historic nature: "retrieve"
* was the PostQUEL keyword for what today is SELECT. "RIR" stands for
* "Retrieve-Instead-Retrieve", that is an ON SELECT DO INSTEAD SELECT rule
* (which has to be unconditional and where only one rule can exist on each
* relation).
*
*-------------------------------------------------------------------------
*/
#include "postgres.h"
#include "access/relation.h"
#include "access/sysattr.h"
#include "access/table.h"
#include "catalog/dependency.h"
#include "catalog/pg_type.h"
#include "commands/trigger.h"
#include "executor/executor.h"
#include "foreign/fdwapi.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
Calculate extraUpdatedCols in query rewriter, not parser. It's unsafe to do this at parse time because addition of generated columns to a table would not invalidate stored rules containing UPDATEs on the table ... but there might now be dependent generated columns that were not there when the rule was made. This also fixes an oversight that rewriteTargetView failed to update extraUpdatedCols when transforming an UPDATE on an updatable view. (Since the new calculation is downstream of that, rewriteTargetView doesn't actually need to do anything; but before, there was a demonstrable bug there.) In v13 and HEAD, this leads to easily-visible bugs because (since commit c6679e4fc) we won't recalculate generated columns that aren't listed in extraUpdatedCols. In v12 this bitmap is mostly just used for trigger-firing decisions, so you'd only notice a problem if a trigger cared whether a generated column had been updated. I'd complained about this back in May, but then forgot about it until bug #16671 from Michael Paul Killian revived the issue. Back-patch to v12 where this field was introduced. If existing stored rules contain any extraUpdatedCols values, they'll be ignored because the rewriter will overwrite them, so the bug will be fixed even for existing rules. (But note that if someone were to update to 13.1 or 12.5, store some rules with UPDATEs on tables having generated columns, and then downgrade to a prior minor version, they might observe issues similar to what this patch fixes. That seems unlikely enough to not be worth going to a lot of effort to fix.) Discussion: https://postgr.es/m/10206.1588964727@sss.pgh.pa.us Discussion: https://postgr.es/m/16671-2fa55851859fb166@postgresql.org
2020-10-28 18:47:02 +01:00
#include "optimizer/optimizer.h"
1999-07-16 07:00:38 +02:00
#include "parser/analyze.h"
#include "parser/parse_coerce.h"
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *. When expanding an updatable view that is an INSERT's target, the rewriter failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally worked if the view was just "SELECT * FROM ...", as the transformation would be a no-op in that case. With more complicated view targetlists, this omission would often lead to "attribute ... has the wrong type" errors or even crashes, as reported by Mario De Frutos Dieguez. Fix by adding code to rewriteTargetView to fix up the data structure correctly. The easiest way to update the exclRelTlist list is to rebuild it from scratch looking at the new target relation, so factor the code for that out of transformOnConflictClause to make it sharable. In passing, avoid duplicate permissions checks against the EXCLUDED pseudo-relation, and prevent useless view expansion of that relation's dummy RTE. The latter is only known to happen (after this patch) in cases where the query would fail later due to not having any INSTEAD OF triggers for the view. But by exactly that token, it would create an unintended and very poorly tested state of the query data structure, so it seems like a good idea to prevent it from happening at all. This has been broken since ON CONFLICT was introduced, so back-patch to 9.5. Dean Rasheed, based on an earlier patch by Amit Langote; comment-kibitzing and back-patching by me Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
#include "parser/parse_relation.h"
#include "parser/parsetree.h"
Changes pg_trigger and extend pg_rewrite in order to allow triggers and rules to be defined with different, per session controllable, behaviors for replication purposes. This will allow replication systems like Slony-I and, as has been stated on pgsql-hackers, other products to control the firing mechanism of triggers and rewrite rules without modifying the system catalog directly. The firing mechanisms are controlled by a new superuser-only GUC variable, session_replication_role, together with a change to pg_trigger.tgenabled and a new column pg_rewrite.ev_enabled. Both columns are a single char data type now (tgenabled was a bool before). The possible values in these attributes are: 'O' - Trigger/Rule fires when session_replication_role is "origin" (default) or "local". This is the default behavior. 'D' - Trigger/Rule is disabled and fires never 'A' - Trigger/Rule fires always regardless of the setting of session_replication_role 'R' - Trigger/Rule fires when session_replication_role is "replica" The GUC variable can only be changed as long as the system does not have any cached query plans. This will prevent changing the session role and accidentally executing stored procedures or functions that have plans cached that expand to the wrong query set due to differences in the rule firing semantics. The SQL syntax for changing a triggers/rules firing semantics is ALTER TABLE <tabname> <when> TRIGGER|RULE <name>; <when> ::= ENABLE | ENABLE ALWAYS | ENABLE REPLICA | DISABLE psql's \d command as well as pg_dump are extended in a backward compatible fashion. Jan
2007-03-20 00:38:32 +01:00
#include "rewrite/rewriteDefine.h"
#include "rewrite/rewriteHandler.h"
1999-07-16 07:00:38 +02:00
#include "rewrite/rewriteManip.h"
#include "rewrite/rewriteSearchCycle.h"
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
#include "rewrite/rowsecurity.h"
#include "utils/builtins.h"
1999-07-16 07:00:38 +02:00
#include "utils/lsyscache.h"
#include "utils/rel.h"
/* We use a list of these to detect recursion in RewriteQuery */
typedef struct rewrite_event
{
Oid relation; /* OID of relation having rules */
CmdType event; /* type of rule being fired */
} rewrite_event;
typedef struct acquireLocksOnSubLinks_context
{
bool for_execute; /* AcquireRewriteLocks' forExecute param */
} acquireLocksOnSubLinks_context;
static bool acquireLocksOnSubLinks(Node *node,
acquireLocksOnSubLinks_context *context);
static Query *rewriteRuleAction(Query *parsetree,
Query *rule_action,
Node *rule_qual,
int rt_index,
CmdType event,
bool *returning_flag);
static List *adjustJoinTreeList(Query *parsetree, bool removert, int rt_index);
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
static List *rewriteTargetListIU(List *targetList,
CmdType commandType,
OverridingKind override,
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
Relation target_relation,
RangeTblEntry *values_rte,
int values_rte_index,
Bitmapset **unused_values_attrnos);
static TargetEntry *process_matched_tle(TargetEntry *src_tle,
TargetEntry *prior_tle,
const char *attrName);
static Node *get_assignment_input(Node *node);
static Bitmapset *findDefaultOnlyColumns(RangeTblEntry *rte);
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
static bool rewriteValuesRTE(Query *parsetree, RangeTblEntry *rte, int rti,
Relation target_relation,
Bitmapset *unused_cols);
static void rewriteValuesRTEToNulls(Query *parsetree, RangeTblEntry *rte);
static void markQueryForLocking(Query *qry, Node *jtnode,
LockClauseStrength strength, LockWaitPolicy waitPolicy,
bool pushedDown);
static List *matchLocks(CmdType event, RuleLock *rulelocks,
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
int varno, Query *parsetree, bool *hasUpdate);
static Query *fireRIRrules(Query *parsetree, List *activeRIRs);
static bool view_has_instead_trigger(Relation view, CmdType event);
static Bitmapset *adjust_view_column_set(Bitmapset *cols, List *targetlist);
/*
* AcquireRewriteLocks -
* Acquire suitable locks on all the relations mentioned in the Query.
* These locks will ensure that the relation schemas don't change under us
* while we are rewriting, planning, and executing the query.
*
* Caution: this may modify the querytree, therefore caller should usually
* have done a copyObject() to make a writable copy of the querytree in the
* current memory context.
*
* forExecute indicates that the query is about to be executed. If so,
* we'll acquire the lock modes specified in the RTE rellockmode fields.
* If forExecute is false, AccessShareLock is acquired on all relations.
* This case is suitable for ruleutils.c, for example, where we only need
* schema stability and we don't intend to actually modify any relations.
*
* forUpdatePushedDown indicates that a pushed-down FOR [KEY] UPDATE/SHARE
* applies to the current subquery, requiring all rels to be opened with at
* least RowShareLock. This should always be false at the top of the
* recursion. When it is true, we adjust RTE rellockmode fields to reflect
* the higher lock level. This flag is ignored if forExecute is false.
*
* A secondary purpose of this routine is to fix up JOIN RTE references to
* dropped columns (see details below). Such RTEs are modified in-place.
*
* This processing can, and for efficiency's sake should, be skipped when the
* querytree has just been built by the parser: parse analysis already got
* all the same locks we'd get here, and the parser will have omitted dropped
* columns from JOINs to begin with. But we must do this whenever we are
* dealing with a querytree produced earlier than the current command.
*
* About JOINs and dropped columns: although the parser never includes an
* already-dropped column in a JOIN RTE's alias var list, it is possible for
* such a list in a stored rule to include references to dropped columns.
* (If the column is not explicitly referenced anywhere else in the query,
* the dependency mechanism won't consider it used by the rule and so won't
* prevent the column drop.) To support get_rte_attribute_is_dropped(), we
* replace join alias vars that reference dropped columns with null pointers.
*
* (In PostgreSQL 8.0, we did not do this processing but instead had
* get_rte_attribute_is_dropped() recurse to detect dropped columns in joins.
* That approach had horrible performance unfortunately; in particular
* construction of a nested join was O(N^2) in the nesting depth.)
*/
void
AcquireRewriteLocks(Query *parsetree,
bool forExecute,
bool forUpdatePushedDown)
{
ListCell *l;
int rt_index;
acquireLocksOnSubLinks_context context;
context.for_execute = forExecute;
/*
* First, process RTEs of the current query level.
*/
rt_index = 0;
foreach(l, parsetree->rtable)
{
RangeTblEntry *rte = (RangeTblEntry *) lfirst(l);
Relation rel;
LOCKMODE lockmode;
List *newaliasvars;
Index curinputvarno;
RangeTblEntry *curinputrte;
ListCell *ll;
++rt_index;
switch (rte->rtekind)
{
case RTE_RELATION:
2005-10-15 04:49:52 +02:00
/*
* Grab the appropriate lock type for the relation, and do not
* release it until end of transaction. This protects the
* rewriter, planner, and executor against schema changes
* mid-query.
*
* If forExecute is false, ignore rellockmode and just use
* AccessShareLock.
*/
if (!forExecute)
lockmode = AccessShareLock;
else if (forUpdatePushedDown)
{
/* Upgrade RTE's lock mode to reflect pushed-down lock */
if (rte->rellockmode == AccessShareLock)
rte->rellockmode = RowShareLock;
lockmode = rte->rellockmode;
}
else
lockmode = rte->rellockmode;
rel = table_open(rte->relid, lockmode);
/*
* While we have the relation open, update the RTE's relkind,
* just in case it changed since this rule was made.
*/
rte->relkind = rel->rd_rel->relkind;
table_close(rel, NoLock);
break;
case RTE_JOIN:
2005-10-15 04:49:52 +02:00
/*
* Scan the join's alias var list to see if any columns have
* been dropped, and if so replace those Vars with null
* pointers.
*
* Since a join has only two inputs, we can expect to see
* multiple references to the same input RTE; optimize away
* multiple fetches.
*/
newaliasvars = NIL;
curinputvarno = 0;
curinputrte = NULL;
foreach(ll, rte->joinaliasvars)
{
Var *aliasitem = (Var *) lfirst(ll);
Var *aliasvar = aliasitem;
/* Look through any implicit coercion */
aliasvar = (Var *) strip_implicit_coercions((Node *) aliasvar);
/*
* If the list item isn't a simple Var, then it must
* represent a merged column, ie a USING column, and so it
* couldn't possibly be dropped, since it's referenced in
* the join clause. (Conceivably it could also be a null
* pointer already? But that's OK too.)
*/
if (aliasvar && IsA(aliasvar, Var))
{
/*
* The elements of an alias list have to refer to
* earlier RTEs of the same rtable, because that's the
* order the planner builds things in. So we already
* processed the referenced RTE, and so it's safe to
* use get_rte_attribute_is_dropped on it. (This might
* not hold after rewriting or planning, but it's OK
* to assume here.)
*/
Assert(aliasvar->varlevelsup == 0);
if (aliasvar->varno != curinputvarno)
{
curinputvarno = aliasvar->varno;
if (curinputvarno >= rt_index)
elog(ERROR, "unexpected varno %d in JOIN RTE %d",
curinputvarno, rt_index);
curinputrte = rt_fetch(curinputvarno,
parsetree->rtable);
}
if (get_rte_attribute_is_dropped(curinputrte,
aliasvar->varattno))
{
/* Replace the join alias item with a NULL */
aliasitem = NULL;
}
}
newaliasvars = lappend(newaliasvars, aliasitem);
}
rte->joinaliasvars = newaliasvars;
break;
case RTE_SUBQUERY:
2005-10-15 04:49:52 +02:00
/*
* The subquery RTE itself is all right, but we have to
* recurse to process the represented subquery.
*/
AcquireRewriteLocks(rte->subquery,
forExecute,
(forUpdatePushedDown ||
get_parse_rowmark(parsetree, rt_index) != NULL));
break;
default:
/* ignore other types of RTEs */
break;
}
}
/* Recurse into subqueries in WITH */
foreach(l, parsetree->cteList)
{
CommonTableExpr *cte = (CommonTableExpr *) lfirst(l);
AcquireRewriteLocks((Query *) cte->ctequery, forExecute, false);
}
/*
* Recurse into sublink subqueries, too. But we already did the ones in
* the rtable and cteList.
*/
if (parsetree->hasSubLinks)
query_tree_walker(parsetree, acquireLocksOnSubLinks, &context,
QTW_IGNORE_RC_SUBQUERIES);
}
/*
* Walker to find sublink subqueries for AcquireRewriteLocks
*/
static bool
acquireLocksOnSubLinks(Node *node, acquireLocksOnSubLinks_context *context)
{
if (node == NULL)
return false;
if (IsA(node, SubLink))
{
SubLink *sub = (SubLink *) node;
/* Do what we came for */
AcquireRewriteLocks((Query *) sub->subselect,
context->for_execute,
false);
/* Fall through to process lefthand args of SubLink */
}
/*
* Do NOT recurse into Query nodes, because AcquireRewriteLocks already
* processed subselects of subselects for us.
*/
return expression_tree_walker(node, acquireLocksOnSubLinks, context);
}
/*
* rewriteRuleAction -
* Rewrite the rule action with appropriate qualifiers (taken from
* the triggering query).
*
* Input arguments:
* parsetree - original query
* rule_action - one action (query) of a rule
* rule_qual - WHERE condition of rule, or NULL if unconditional
* rt_index - RT index of result relation in original query
* event - type of rule event
* Output arguments:
* *returning_flag - set true if we rewrite RETURNING clause in rule_action
* (must be initialized to false)
* Return value:
* rewritten form of rule_action
*/
static Query *
rewriteRuleAction(Query *parsetree,
Query *rule_action,
Node *rule_qual,
int rt_index,
CmdType event,
bool *returning_flag)
{
int current_varno,
new_varno;
int rt_length;
Query *sub_action;
Query **sub_action_ptr;
acquireLocksOnSubLinks_context context;
ListCell *lc;
context.for_execute = true;
/*
* Make modifiable copies of rule action and qual (what we're passed are
* the stored versions in the relcache; don't touch 'em!).
*/
rule_action = copyObject(rule_action);
rule_qual = copyObject(rule_qual);
/*
* Acquire necessary locks and fix any deleted JOIN RTE entries.
*/
AcquireRewriteLocks(rule_action, true, false);
(void) acquireLocksOnSubLinks(rule_qual, &context);
current_varno = rt_index;
rt_length = list_length(parsetree->rtable);
new_varno = PRS2_NEW_VARNO + rt_length;
/*
* Adjust rule action and qual to offset its varnos, so that we can merge
* its rtable with the main parsetree's rtable.
*
* If the rule action is an INSERT...SELECT, the OLD/NEW rtable entries
* will be in the SELECT part, and we have to modify that rather than the
* top-level INSERT (kluge!).
*/
sub_action = getInsertSelectQuery(rule_action, &sub_action_ptr);
OffsetVarNodes((Node *) sub_action, rt_length, 0);
OffsetVarNodes(rule_qual, rt_length, 0);
/* but references to OLD should point at original rt_index */
ChangeVarNodes((Node *) sub_action,
PRS2_OLD_VARNO + rt_length, rt_index, 0);
ChangeVarNodes(rule_qual,
PRS2_OLD_VARNO + rt_length, rt_index, 0);
/*
* Mark any subquery RTEs in the rule action as LATERAL if they contain
* Vars referring to the current query level (references to NEW/OLD).
* Those really are lateral references, but we've historically not
* required users to mark such subqueries with LATERAL explicitly. But
* the planner will complain if such Vars exist in a non-LATERAL subquery,
* so we have to fix things up here.
*/
foreach(lc, sub_action->rtable)
{
RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc);
if (rte->rtekind == RTE_SUBQUERY && !rte->lateral &&
contain_vars_of_level((Node *) rte->subquery, 1))
rte->lateral = true;
}
/*
* Generate expanded rtable consisting of main parsetree's rtable plus
* rule action's rtable; this becomes the complete rtable for the rule
* action. Some of the entries may be unused after we finish rewriting,
* but we leave them all in place for two reasons:
*
* We'd have a much harder job to adjust the query's varnos if we
* selectively removed RT entries.
*
* If the rule is INSTEAD, then the original query won't be executed at
* all, and so its rtable must be preserved so that the executor will do
* the correct permissions checks on it.
*
* RT entries that are not referenced in the completed jointree will be
* ignored by the planner, so they do not affect query semantics. But any
* permissions checks specified in them will be applied during executor
* startup (see ExecCheckRTEPerms()). This allows us to check that the
* caller has, say, insert-permission on a view, when the view is not
* semantically referenced at all in the resulting query.
*
* When a rule is not INSTEAD, the permissions checks done on its copied
* RT entries will be redundant with those done during execution of the
* original query, but we don't bother to treat that case differently.
*
* NOTE: because planner will destructively alter rtable, we must ensure
* that rule action's rtable is separate and shares no substructure with
* the main rtable. Hence do a deep copy here.
*
* Note also that RewriteQuery() relies on the fact that RT entries from
* the original query appear at the start of the expanded rtable, so
* beware of changing this.
*/
sub_action->rtable = list_concat(copyObject(parsetree->rtable),
sub_action->rtable);
/*
* There could have been some SubLinks in parsetree's rtable, in which
* case we'd better mark the sub_action correctly.
*/
if (parsetree->hasSubLinks && !sub_action->hasSubLinks)
{
foreach(lc, parsetree->rtable)
{
RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc);
switch (rte->rtekind)
{
Redesign tablesample method API, and do extensive code review. The original implementation of TABLESAMPLE modeled the tablesample method API on index access methods, which wasn't a good choice because, without specialized DDL commands, there's no way to build an extension that can implement a TSM. (Raw inserts into system catalogs are not an acceptable thing to do, because we can't undo them during DROP EXTENSION, nor will pg_upgrade behave sanely.) Instead adopt an API more like procedural language handlers or foreign data wrappers, wherein the only SQL-level support object needed is a single handler function identified by having a special return type. This lets us get rid of the supporting catalog altogether, so that no custom DDL support is needed for the feature. Adjust the API so that it can support non-constant tablesample arguments (the original coding assumed we could evaluate the argument expressions at ExecInitSampleScan time, which is undesirable even if it weren't outright unsafe), and discourage sampling methods from looking at invisible tuples. Make sure that the BERNOULLI and SYSTEM methods are genuinely repeatable within and across queries, as required by the SQL standard, and deal more honestly with methods that can't support that requirement. Make a full code-review pass over the tablesample additions, and fix assorted bugs, omissions, infelicities, and cosmetic issues (such as failure to put the added code stanzas in a consistent ordering). Improve EXPLAIN's output of tablesample plans, too. Back-patch to 9.5 so that we don't have to support the original API in production.
2015-07-25 20:39:00 +02:00
case RTE_RELATION:
sub_action->hasSubLinks =
checkExprHasSubLink((Node *) rte->tablesample);
break;
case RTE_FUNCTION:
sub_action->hasSubLinks =
checkExprHasSubLink((Node *) rte->functions);
break;
case RTE_TABLEFUNC:
sub_action->hasSubLinks =
checkExprHasSubLink((Node *) rte->tablefunc);
break;
case RTE_VALUES:
sub_action->hasSubLinks =
checkExprHasSubLink((Node *) rte->values_lists);
break;
default:
/* other RTE types don't contain bare expressions */
break;
}
sub_action->hasSubLinks |=
checkExprHasSubLink((Node *) rte->securityQuals);
if (sub_action->hasSubLinks)
break; /* no need to keep scanning rtable */
}
}
/*
* Also, we might have absorbed some RTEs with RLS conditions into the
* sub_action. If so, mark it as hasRowSecurity, whether or not those
* RTEs will be referenced after we finish rewriting. (Note: currently
* this is a no-op because RLS conditions aren't added till later, but it
* seems like good future-proofing to do this anyway.)
*/
sub_action->hasRowSecurity |= parsetree->hasRowSecurity;
/*
* Each rule action's jointree should be the main parsetree's jointree
* plus that rule's jointree, but usually *without* the original rtindex
* that we're replacing (if present, which it won't be for INSERT). Note
* that if the rule action refers to OLD, its jointree will add a
* reference to rt_index. If the rule action doesn't refer to OLD, but
* either the rule_qual or the user query quals do, then we need to keep
* the original rtindex in the jointree to provide data for the quals. We
* don't want the original rtindex to be joined twice, however, so avoid
* keeping it if the rule action mentions it.
*
* As above, the action's jointree must not share substructure with the
* main parsetree's.
*/
if (sub_action->commandType != CMD_UTILITY)
{
bool keeporig;
List *newjointree;
Assert(sub_action->jointree != NULL);
keeporig = (!rangeTableEntry_used((Node *) sub_action->jointree,
rt_index, 0)) &&
(rangeTableEntry_used(rule_qual, rt_index, 0) ||
rangeTableEntry_used(parsetree->jointree->quals, rt_index, 0));
newjointree = adjustJoinTreeList(parsetree, !keeporig, rt_index);
if (newjointree != NIL)
{
/*
* If sub_action is a setop, manipulating its jointree will do no
* good at all, because the jointree is dummy. (Perhaps someday
* we could push the joining and quals down to the member
* statements of the setop?)
*/
if (sub_action->setOperations != NULL)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("conditional UNION/INTERSECT/EXCEPT statements are not implemented")));
sub_action->jointree->fromlist =
list_concat(newjointree, sub_action->jointree->fromlist);
/*
* There could have been some SubLinks in newjointree, in which
* case we'd better mark the sub_action correctly.
*/
if (parsetree->hasSubLinks && !sub_action->hasSubLinks)
sub_action->hasSubLinks =
checkExprHasSubLink((Node *) newjointree);
}
}
/*
* If the original query has any CTEs, copy them into the rule action. But
* we don't need them for a utility action.
*/
if (parsetree->cteList != NIL && sub_action->commandType != CMD_UTILITY)
{
/*
* Annoying implementation restriction: because CTEs are identified by
* name within a cteList, we can't merge a CTE from the original query
* if it has the same name as any CTE in the rule action.
*
* This could possibly be fixed by using some sort of internally
* generated ID, instead of names, to link CTE RTEs to their CTEs.
Fix rewriter to set hasModifyingCTE correctly on rewritten queries. If we copy data-modifying CTEs from the original query to a replacement query (from a DO INSTEAD rule), we must set hasModifyingCTE properly in the replacement query. Failure to do this can cause various unpleasantness, such as unsafe usage of parallel plans. The code also neglected to propagate hasRecursive, though that's only cosmetic at the moment. A difficulty arises if the rule action is an INSERT...SELECT. We attach the original query's RTEs and CTEs to the sub-SELECT Query, but data-modifying CTEs are only allowed to appear in the topmost Query. For the moment, throw an error in such cases. It would probably be possible to avoid this error by attaching the CTEs to the top INSERT Query instead; but that would require a bunch of new code to adjust ctelevelsup references. Given the narrowness of the use-case, and the need to back-patch this fix, it does not seem worth the trouble for now. We can revisit this if we get field complaints. Per report from Greg Nancarrow. Back-patch to all supported branches. (The test case added here does not fail before v10, but there are plenty of places checking top-level hasModifyingCTE in 9.6, so I have no doubt that this code change is necessary there too.) Greg Nancarrow and Tom Lane Discussion: https://postgr.es/m/CAJcOf-f68DT=26YAMz_i0+Au3TcLO5oiHY5=fL6Sfuits6r+_w@mail.gmail.com Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
2021-09-08 18:05:43 +02:00
* However, decompiling the results would be quite confusing; note the
* merge of hasRecursive flags below, which could change the apparent
* semantics of such redundantly-named CTEs.
*/
foreach(lc, parsetree->cteList)
{
CommonTableExpr *cte = (CommonTableExpr *) lfirst(lc);
ListCell *lc2;
foreach(lc2, sub_action->cteList)
{
CommonTableExpr *cte2 = (CommonTableExpr *) lfirst(lc2);
if (strcmp(cte->ctename, cte2->ctename) == 0)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("WITH query name \"%s\" appears in both a rule action and the query being rewritten",
cte->ctename)));
}
}
/* OK, it's safe to combine the CTE lists */
sub_action->cteList = list_concat(sub_action->cteList,
copyObject(parsetree->cteList));
Fix rewriter to set hasModifyingCTE correctly on rewritten queries. If we copy data-modifying CTEs from the original query to a replacement query (from a DO INSTEAD rule), we must set hasModifyingCTE properly in the replacement query. Failure to do this can cause various unpleasantness, such as unsafe usage of parallel plans. The code also neglected to propagate hasRecursive, though that's only cosmetic at the moment. A difficulty arises if the rule action is an INSERT...SELECT. We attach the original query's RTEs and CTEs to the sub-SELECT Query, but data-modifying CTEs are only allowed to appear in the topmost Query. For the moment, throw an error in such cases. It would probably be possible to avoid this error by attaching the CTEs to the top INSERT Query instead; but that would require a bunch of new code to adjust ctelevelsup references. Given the narrowness of the use-case, and the need to back-patch this fix, it does not seem worth the trouble for now. We can revisit this if we get field complaints. Per report from Greg Nancarrow. Back-patch to all supported branches. (The test case added here does not fail before v10, but there are plenty of places checking top-level hasModifyingCTE in 9.6, so I have no doubt that this code change is necessary there too.) Greg Nancarrow and Tom Lane Discussion: https://postgr.es/m/CAJcOf-f68DT=26YAMz_i0+Au3TcLO5oiHY5=fL6Sfuits6r+_w@mail.gmail.com Discussion: https://postgr.es/m/CAJcOf-fAdj=nDKMsRhQzndm-O13NY4dL6xGcEvdX5Xvbbi0V7g@mail.gmail.com
2021-09-08 18:05:43 +02:00
/* ... and don't forget about the associated flags */
sub_action->hasRecursive |= parsetree->hasRecursive;
sub_action->hasModifyingCTE |= parsetree->hasModifyingCTE;
/*
* If rule_action is different from sub_action (i.e., the rule action
* is an INSERT...SELECT), then we might have just added some
* data-modifying CTEs that are not at the top query level. This is
* disallowed by the parser and we mustn't generate such trees here
* either, so throw an error.
*
* Conceivably such cases could be supported by attaching the original
* query's CTEs to rule_action not sub_action. But to do that, we'd
* have to increment ctelevelsup in RTEs and SubLinks copied from the
* original query. For now, it doesn't seem worth the trouble.
*/
if (sub_action->hasModifyingCTE && rule_action != sub_action)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("INSERT...SELECT rule actions are not supported for queries having data-modifying statements in WITH")));
}
/*
* Event Qualification forces copying of parsetree and splitting into two
* queries one w/rule_qual, one w/NOT rule_qual. Also add user query qual
* onto rule action
*/
AddQual(sub_action, rule_qual);
AddQual(sub_action, parsetree->jointree->quals);
/*
* Rewrite new.attribute with right hand side of target-list entry for
* appropriate field name in insert/update.
*
* KLUGE ALERT: since ReplaceVarsFromTargetList returns a mutated copy, we
* can't just apply it to sub_action; we have to remember to update the
* sublink inside rule_action, too.
*/
if ((event == CMD_INSERT || event == CMD_UPDATE) &&
sub_action->commandType != CMD_UTILITY)
{
sub_action = (Query *)
ReplaceVarsFromTargetList((Node *) sub_action,
new_varno,
0,
rt_fetch(new_varno, sub_action->rtable),
parsetree->targetList,
(event == CMD_UPDATE) ?
REPLACEVARS_CHANGE_VARNO :
REPLACEVARS_SUBSTITUTE_NULL,
current_varno,
NULL);
if (sub_action_ptr)
*sub_action_ptr = sub_action;
else
rule_action = sub_action;
}
/*
* If rule_action has a RETURNING clause, then either throw it away if the
* triggering query has no RETURNING clause, or rewrite it to emit what
* the triggering query's RETURNING clause asks for. Throw an error if
* more than one rule has a RETURNING clause.
*/
if (!parsetree->returningList)
rule_action->returningList = NIL;
else if (rule_action->returningList)
{
if (*returning_flag)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot have RETURNING lists in multiple rules")));
*returning_flag = true;
rule_action->returningList = (List *)
ReplaceVarsFromTargetList((Node *) parsetree->returningList,
parsetree->resultRelation,
0,
rt_fetch(parsetree->resultRelation,
parsetree->rtable),
rule_action->returningList,
REPLACEVARS_REPORT_ERROR,
0,
&rule_action->hasSubLinks);
/*
* There could have been some SubLinks in parsetree's returningList,
* in which case we'd better mark the rule_action correctly.
*/
if (parsetree->hasSubLinks && !rule_action->hasSubLinks)
rule_action->hasSubLinks =
checkExprHasSubLink((Node *) rule_action->returningList);
}
return rule_action;
}
/*
* Copy the query's jointree list, and optionally attempt to remove any
* occurrence of the given rt_index as a top-level join item (we do not look
* for it within join items; this is OK because we are only expecting to find
* it as an UPDATE or DELETE target relation, which will be at the top level
* of the join). Returns modified jointree list --- this is a separate copy
* sharing no nodes with the original.
*/
static List *
adjustJoinTreeList(Query *parsetree, bool removert, int rt_index)
{
List *newjointree = copyObject(parsetree->jointree->fromlist);
ListCell *l;
if (removert)
{
foreach(l, newjointree)
{
RangeTblRef *rtr = lfirst(l);
if (IsA(rtr, RangeTblRef) &&
rtr->rtindex == rt_index)
{
newjointree = foreach_delete_current(newjointree, l);
break;
}
}
}
return newjointree;
}
This is the final state of the rule system for 6.4 after the patch is applied: Rewrite rules on relation level work fine now. Event qualifications on insert/update/delete rules work fine now. I added the new keyword OLD to reference the CURRENT tuple. CURRENT will be removed in 6.5. Update rules can reference NEW and OLD in the rule qualification and the actions. Insert/update/delete rules on views can be established to let them behave like real tables. For insert/update/delete rules multiple actions are supported now. The actions can also be surrounded by parantheses to make psql happy. Multiple actions are required if update to a view requires updates to multiple tables. Regular users are permitted to create/drop rules on tables they have RULE permissions for (DefineQueryRewrite() is now able to get around the access restrictions on pg_rewrite). This enables view creation for regular users too. This required an extra boolean parameter to pg_parse_and_plan() that tells to set skipAcl on all rangetable entries of the resulting queries. There is a new function pg_exec_query_acl_override() that could be used by backend utilities to use this facility. All rule actions (not only views) inherit the permissions of the event relations owner. Sample: User A creates tables T1 and T2, creates rules that log INSERT/UPDATE/DELETE on T1 in T2 (like in the regression tests for rules I created) and grants ALL but RULE on T1 to user B. User B can now fully access T1 and the logging happens in T2. But user B cannot access T2 at all, only the rule actions can. And due to missing RULE permissions on T1, user B cannot disable logging. Rules on the attribute level are disabled (they don't work properly and since regular users are now permitted to create rules I decided to disable them). Rules on select must have exactly one action that is a select (so select rules must be a view definition). UPDATE NEW/OLD rules are disabled (still broken, but triggers can do it). There are two new system views (pg_rule and pg_view) that show the definition of the rules or views so the db admin can see what the users do. They use two new functions pg_get_ruledef() and pg_get_viewdef() that are builtins. The functions pg_get_ruledef() and pg_get_viewdef() could be used to implement rule and view support in pg_dump. PostgreSQL is now the only database system I know, that has rewrite rules on the query level. All others (where I found a rule statement at all) use stored database procedures or the like (triggers as we call them) for active rules (as some call them). Future of the rule system: The now disabled parts of the rule system (attribute level, multiple actions on select and update new stuff) require a complete new rewrite handler from scratch. The old one is too badly wired up. After 6.4 I'll start to work on a new rewrite handler, that fully supports the attribute level rules, multiple actions on select and update new. This will be available for 6.5 so we get full rewrite rule capabilities. Jan
1998-08-24 03:38:11 +02:00
/*
* rewriteTargetListIU - rewrite INSERT/UPDATE targetlist into standard form
*
* This has the following responsibilities:
*
* 1. For an INSERT, add tlist entries to compute default values for any
* attributes that have defaults and are not assigned to in the given tlist.
* (We do not insert anything for default-less attributes, however. The
* planner will later insert NULLs for them, but there's no reason to slow
* down rewriter processing with extra tlist nodes.) Also, for both INSERT
* and UPDATE, replace explicit DEFAULT specifications with column default
* expressions.
*
Remove rewriteTargetListIU's expansion of view targetlists in UPDATE. Commit 2ec993a7c, which added triggers on views, modified the rewriter to add dummy entries like "SET x = x" for all columns that weren't actually being updated by the user in any UPDATE directed at a view. That was needed at the time to produce a complete "NEW" row to pass to the trigger. Later it was found to cause problems for ordinary updatable views, so commit cab5dc5da restricted it to happen only for trigger-updatable views. But in the wake of commit 86dc90056, we really don't need it at all. nodeModifyTable.c populates the trigger "OLD" row from the whole-row variable that is generated for the view, and then it computes the "NEW" row using that old row and the UPDATE targetlist. So there is no need for the UPDATE tlist to have dummy entries, any more than it needs them for regular tables or other types of views. (The comments for rewriteTargetListIU suggest that we must do this for correct expansion of NEW references in rules, but I now think that that was just lazy comment editing in 2ec993a7c. If we didn't need it for rules on views before there were triggers, we don't need it after that.) This essentially propagates 86dc90056's decision that we don't need dummy column updates into the view case. Aside from making the different cases more uniform and hence possibly forestalling future bugs, it ought to save a little bit of rewriter/planner effort. Discussion: https://postgr.es/m/2181213.1619397634@sss.pgh.pa.us
2021-04-26 19:58:00 +02:00
* 2. Merge multiple entries for the same target attribute, or declare error
* if we can't. Multiple entries are only allowed for INSERT/UPDATE of
* portions of an array or record field, for example
* UPDATE table SET foo[2] = 42, foo[4] = 43;
* We can merge such operations into a single assignment op. Essentially,
* the expression we want to produce in this case is like
* foo = array_set_element(array_set_element(foo, 2, 42), 4, 43)
*
Remove rewriteTargetListIU's expansion of view targetlists in UPDATE. Commit 2ec993a7c, which added triggers on views, modified the rewriter to add dummy entries like "SET x = x" for all columns that weren't actually being updated by the user in any UPDATE directed at a view. That was needed at the time to produce a complete "NEW" row to pass to the trigger. Later it was found to cause problems for ordinary updatable views, so commit cab5dc5da restricted it to happen only for trigger-updatable views. But in the wake of commit 86dc90056, we really don't need it at all. nodeModifyTable.c populates the trigger "OLD" row from the whole-row variable that is generated for the view, and then it computes the "NEW" row using that old row and the UPDATE targetlist. So there is no need for the UPDATE tlist to have dummy entries, any more than it needs them for regular tables or other types of views. (The comments for rewriteTargetListIU suggest that we must do this for correct expansion of NEW references in rules, but I now think that that was just lazy comment editing in 2ec993a7c. If we didn't need it for rules on views before there were triggers, we don't need it after that.) This essentially propagates 86dc90056's decision that we don't need dummy column updates into the view case. Aside from making the different cases more uniform and hence possibly forestalling future bugs, it ought to save a little bit of rewriter/planner effort. Discussion: https://postgr.es/m/2181213.1619397634@sss.pgh.pa.us
2021-04-26 19:58:00 +02:00
* 3. Sort the tlist into standard order: non-junk fields in order by resno,
* then junk fields (these in no particular order).
*
Remove rewriteTargetListIU's expansion of view targetlists in UPDATE. Commit 2ec993a7c, which added triggers on views, modified the rewriter to add dummy entries like "SET x = x" for all columns that weren't actually being updated by the user in any UPDATE directed at a view. That was needed at the time to produce a complete "NEW" row to pass to the trigger. Later it was found to cause problems for ordinary updatable views, so commit cab5dc5da restricted it to happen only for trigger-updatable views. But in the wake of commit 86dc90056, we really don't need it at all. nodeModifyTable.c populates the trigger "OLD" row from the whole-row variable that is generated for the view, and then it computes the "NEW" row using that old row and the UPDATE targetlist. So there is no need for the UPDATE tlist to have dummy entries, any more than it needs them for regular tables or other types of views. (The comments for rewriteTargetListIU suggest that we must do this for correct expansion of NEW references in rules, but I now think that that was just lazy comment editing in 2ec993a7c. If we didn't need it for rules on views before there were triggers, we don't need it after that.) This essentially propagates 86dc90056's decision that we don't need dummy column updates into the view case. Aside from making the different cases more uniform and hence possibly forestalling future bugs, it ought to save a little bit of rewriter/planner effort. Discussion: https://postgr.es/m/2181213.1619397634@sss.pgh.pa.us
2021-04-26 19:58:00 +02:00
* We must do items 1 and 2 before firing rewrite rules, else rewritten
* references to NEW.foo will produce wrong or incomplete results. Item 3
Rework planning and execution of UPDATE and DELETE. This patch makes two closely related sets of changes: 1. For UPDATE, the subplan of the ModifyTable node now only delivers the new values of the changed columns (i.e., the expressions computed in the query's SET clause) plus row identity information such as CTID. ModifyTable must re-fetch the original tuple to merge in the old values of any unchanged columns. The core advantage of this is that the changed columns are uniform across all tables of an inherited or partitioned target relation, whereas the other columns might not be. A secondary advantage, when the UPDATE involves joins, is that less data needs to pass through the plan tree. The disadvantage of course is an extra fetch of each tuple to be updated. However, that seems to be very nearly free in context; even worst-case tests don't show it to add more than a couple percent to the total query cost. At some point it might be interesting to combine the re-fetch with the tuple access that ModifyTable must do anyway to mark the old tuple dead; but that would require a good deal of refactoring and it seems it wouldn't buy all that much, so this patch doesn't attempt it. 2. For inherited UPDATE/DELETE, instead of generating a separate subplan for each target relation, we now generate a single subplan that is just exactly like a SELECT's plan, then stick ModifyTable on top of that. To let ModifyTable know which target relation a given incoming row refers to, a tableoid junk column is added to the row identity information. This gets rid of the horrid hack that was inheritance_planner(), eliminating O(N^2) planning cost and memory consumption in cases where there were many unprunable target relations. Point 2 of course requires point 1, so that there is a uniform definition of the non-junk columns to be returned by the subplan. We can't insist on uniform definition of the row identity junk columns however, if we want to keep the ability to have both plain and foreign tables in a partitioning hierarchy. Since it wouldn't scale very far to have every child table have its own row identity column, this patch includes provisions to merge similar row identity columns into one column of the subplan result. In particular, we can merge the whole-row Vars typically used as row identity by FDWs into one column by pretending they are type RECORD. (It's still okay for the actual composite Datums to be labeled with the table's rowtype OID, though.) There is more that can be done to file down residual inefficiencies in this patch, but it seems to be committable now. FDW authors should note several API changes: * The argument list for AddForeignUpdateTargets() has changed, and so has the method it must use for adding junk columns to the query. Call add_row_identity_var() instead of manipulating the parse tree directly. You might want to reconsider exactly what you're adding, too. * PlanDirectModify() must now work a little harder to find the ForeignScan plan node; if the foreign table is part of a partitioning hierarchy then the ForeignScan might not be the direct child of ModifyTable. See postgres_fdw for sample code. * To check whether a relation is a target relation, it's no longer sufficient to compare its relid to root->parse->resultRelation. Instead, check it against all_result_relids or leaf_result_relids, as appropriate. Amit Langote and Tom Lane Discussion: https://postgr.es/m/CA+HiwqHpHdqdDn48yCEhynnniahH78rwcrv1rEX65-fsZGBOLQ@mail.gmail.com
2021-03-31 17:52:34 +02:00
* is not needed for rewriting, but it is helpful for the planner, and we
* can do it essentially for free while handling the other items.
*
* If values_rte is non-NULL (i.e., we are doing a multi-row INSERT using
* values from a VALUES RTE), we populate *unused_values_attrnos with the
* attribute numbers of any unused columns from the VALUES RTE. This can
* happen for identity and generated columns whose targetlist entries are
* replaced with generated expressions (if INSERT ... OVERRIDING USER VALUE is
* used, or all the values to be inserted are DEFAULT). This information is
* required by rewriteValuesRTE() to handle any DEFAULT items in the unused
* columns. The caller must have initialized *unused_values_attrnos to NULL.
*/
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
static List *
rewriteTargetListIU(List *targetList,
CmdType commandType,
OverridingKind override,
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
Relation target_relation,
RangeTblEntry *values_rte,
int values_rte_index,
Bitmapset **unused_values_attrnos)
{
TargetEntry **new_tles;
List *new_tlist = NIL;
List *junk_tlist = NIL;
Form_pg_attribute att_tup;
int attrno,
next_junk_attrno,
numattrs;
ListCell *temp;
Bitmapset *default_only_cols = NULL;
/*
* We process the normal (non-junk) attributes by scanning the input tlist
* once and transferring TLEs into an array, then scanning the array to
* build an output tlist. This avoids O(N^2) behavior for large numbers
* of attributes.
*
* Junk attributes are tossed into a separate list during the same tlist
* scan, then appended to the reconstructed tlist.
*/
numattrs = RelationGetNumberOfAttributes(target_relation);
new_tles = (TargetEntry **) palloc0(numattrs * sizeof(TargetEntry *));
next_junk_attrno = numattrs + 1;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
foreach(temp, targetList)
{
TargetEntry *old_tle = (TargetEntry *) lfirst(temp);
if (!old_tle->resjunk)
{
/* Normal attr: stash it into new_tles[] */
attrno = old_tle->resno;
if (attrno < 1 || attrno > numattrs)
elog(ERROR, "bogus resno %d in targetlist", attrno);
att_tup = TupleDescAttr(target_relation->rd_att, attrno - 1);
/* We can (and must) ignore deleted attributes */
if (att_tup->attisdropped)
continue;
/* Merge with any prior assignment to same attribute */
new_tles[attrno - 1] =
process_matched_tle(old_tle,
new_tles[attrno - 1],
NameStr(att_tup->attname));
}
else
{
/*
* Copy all resjunk tlist entries to junk_tlist, and assign them
* resnos above the last real resno.
*
* Typical junk entries include ORDER BY or GROUP BY expressions
* (are these actually possible in an INSERT or UPDATE?), system
* attribute references, etc.
*/
/* Get the resno right, but don't copy unnecessarily */
if (old_tle->resno != next_junk_attrno)
{
old_tle = flatCopyTargetEntry(old_tle);
old_tle->resno = next_junk_attrno;
}
junk_tlist = lappend(junk_tlist, old_tle);
next_junk_attrno++;
}
}
for (attrno = 1; attrno <= numattrs; attrno++)
{
TargetEntry *new_tle = new_tles[attrno - 1];
bool apply_default;
att_tup = TupleDescAttr(target_relation->rd_att, attrno - 1);
/* We can (and must) ignore deleted attributes */
if (att_tup->attisdropped)
continue;
/*
* Handle the two cases where we need to insert a default expression:
* it's an INSERT and there's no tlist entry for the column, or the
* tlist entry is a DEFAULT placeholder node.
*/
apply_default = ((new_tle == NULL && commandType == CMD_INSERT) ||
(new_tle && new_tle->expr && IsA(new_tle->expr, SetToDefault)));
if (commandType == CMD_INSERT)
{
int values_attrno = 0;
/* Source attribute number for values that come from a VALUES RTE */
if (values_rte && new_tle && IsA(new_tle->expr, Var))
{
Var *var = (Var *) new_tle->expr;
if (var->varno == values_rte_index)
values_attrno = var->varattno;
}
/*
* Can only insert DEFAULT into GENERATED ALWAYS identity columns,
* unless either OVERRIDING USER VALUE or OVERRIDING SYSTEM VALUE
* is specified.
*/
if (att_tup->attidentity == ATTRIBUTE_IDENTITY_ALWAYS && !apply_default)
{
if (override == OVERRIDING_USER_VALUE)
apply_default = true;
else if (override != OVERRIDING_SYSTEM_VALUE)
{
/*
* If this column's values come from a VALUES RTE, test
* whether it contains only SetToDefault items. Since the
* VALUES list might be quite large, we arrange to only
* scan it once.
*/
if (values_attrno != 0)
{
if (default_only_cols == NULL)
default_only_cols = findDefaultOnlyColumns(values_rte);
if (bms_is_member(values_attrno, default_only_cols))
apply_default = true;
}
if (!apply_default)
ereport(ERROR,
(errcode(ERRCODE_GENERATED_ALWAYS),
errmsg("cannot insert a non-DEFAULT value into column \"%s\"",
NameStr(att_tup->attname)),
errdetail("Column \"%s\" is an identity column defined as GENERATED ALWAYS.",
NameStr(att_tup->attname)),
errhint("Use OVERRIDING SYSTEM VALUE to override.")));
}
}
/*
* Although inserting into a GENERATED BY DEFAULT identity column
* is allowed, apply the default if OVERRIDING USER VALUE is
* specified.
*/
if (att_tup->attidentity == ATTRIBUTE_IDENTITY_BY_DEFAULT &&
override == OVERRIDING_USER_VALUE)
apply_default = true;
/*
* Can only insert DEFAULT into generated columns, regardless of
* any OVERRIDING clauses.
*/
if (att_tup->attgenerated && !apply_default)
{
/*
* If this column's values come from a VALUES RTE, test
* whether it contains only SetToDefault items, as above.
*/
if (values_attrno != 0)
{
if (default_only_cols == NULL)
default_only_cols = findDefaultOnlyColumns(values_rte);
if (bms_is_member(values_attrno, default_only_cols))
apply_default = true;
}
if (!apply_default)
ereport(ERROR,
(errcode(ERRCODE_GENERATED_ALWAYS),
errmsg("cannot insert a non-DEFAULT value into column \"%s\"",
NameStr(att_tup->attname)),
errdetail("Column \"%s\" is a generated column.",
NameStr(att_tup->attname))));
}
/*
* For an INSERT from a VALUES RTE, return the attribute numbers
* of any VALUES columns that will no longer be used (due to the
* targetlist entry being replaced by a default expression).
*/
if (values_attrno != 0 && apply_default && unused_values_attrnos)
*unused_values_attrnos = bms_add_member(*unused_values_attrnos,
values_attrno);
}
/*
* Updates to identity and generated columns follow the same rules as
* above, except that UPDATE doesn't admit OVERRIDING clauses. Also,
* the source can't be a VALUES RTE, so we needn't consider that.
*/
if (commandType == CMD_UPDATE)
{
if (att_tup->attidentity == ATTRIBUTE_IDENTITY_ALWAYS &&
new_tle && !apply_default)
ereport(ERROR,
(errcode(ERRCODE_GENERATED_ALWAYS),
errmsg("column \"%s\" can only be updated to DEFAULT",
NameStr(att_tup->attname)),
errdetail("Column \"%s\" is an identity column defined as GENERATED ALWAYS.",
NameStr(att_tup->attname))));
if (att_tup->attgenerated && new_tle && !apply_default)
ereport(ERROR,
(errcode(ERRCODE_GENERATED_ALWAYS),
errmsg("column \"%s\" can only be updated to DEFAULT",
NameStr(att_tup->attname)),
errdetail("Column \"%s\" is a generated column.",
NameStr(att_tup->attname))));
}
if (att_tup->attgenerated)
{
/*
* stored generated column will be fixed in executor
*/
new_tle = NULL;
}
else if (apply_default)
{
Node *new_expr;
new_expr = build_column_default(target_relation, attrno);
/*
* If there is no default (ie, default is effectively NULL), we
* can omit the tlist entry in the INSERT case, since the planner
* can insert a NULL for itself, and there's no point in spending
* any more rewriter cycles on the entry. But in the UPDATE case
* we've got to explicitly set the column to NULL.
*/
if (!new_expr)
{
if (commandType == CMD_INSERT)
new_tle = NULL;
else
{
new_expr = (Node *) makeConst(att_tup->atttypid,
-1,
att_tup->attcollation,
att_tup->attlen,
(Datum) 0,
true, /* isnull */
att_tup->attbyval);
/* this is to catch a NOT NULL domain constraint */
new_expr = coerce_to_domain(new_expr,
InvalidOid, -1,
att_tup->atttypid,
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
COERCION_IMPLICIT,
COERCE_IMPLICIT_CAST,
-1,
false);
}
}
if (new_expr)
new_tle = makeTargetEntry((Expr *) new_expr,
attrno,
pstrdup(NameStr(att_tup->attname)),
false);
}
if (new_tle)
new_tlist = lappend(new_tlist, new_tle);
}
pfree(new_tles);
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
return list_concat(new_tlist, junk_tlist);
}
/*
* Convert a matched TLE from the original tlist into a correct new TLE.
*
* This routine detects and handles multiple assignments to the same target
* attribute. (The attribute name is needed only for error messages.)
*/
static TargetEntry *
process_matched_tle(TargetEntry *src_tle,
TargetEntry *prior_tle,
const char *attrName)
{
TargetEntry *result;
CoerceToDomain *coerce_expr = NULL;
Node *src_expr;
Node *prior_expr;
Node *src_input;
Node *prior_input;
Node *priorbottom;
Node *newexpr;
if (prior_tle == NULL)
{
/*
* Normal case where this is the first assignment to the attribute.
*/
return src_tle;
}
/*----------
* Multiple assignments to same attribute. Allow only if all are
* FieldStore or SubscriptingRef assignment operations. This is a bit
* tricky because what we may actually be looking at is a nest of
* such nodes; consider
* UPDATE tab SET col.fld1.subfld1 = x, col.fld2.subfld2 = y
* The two expressions produced by the parser will look like
* FieldStore(col, fld1, FieldStore(placeholder, subfld1, x))
2014-10-03 17:47:27 +02:00
* FieldStore(col, fld2, FieldStore(placeholder, subfld2, y))
* However, we can ignore the substructure and just consider the top
* FieldStore or SubscriptingRef from each assignment, because it works to
* combine these as
* FieldStore(FieldStore(col, fld1,
* FieldStore(placeholder, subfld1, x)),
2014-10-03 17:47:27 +02:00
* fld2, FieldStore(placeholder, subfld2, y))
* Note the leftmost expression goes on the inside so that the
* assignments appear to occur left-to-right.
*
* For FieldStore, instead of nesting we can generate a single
* FieldStore with multiple target fields. We must nest when
* SubscriptingRefs are involved though.
*
* As a further complication, the destination column might be a domain,
* resulting in each assignment containing a CoerceToDomain node over a
* FieldStore or SubscriptingRef. These should have matching target
* domains, so we strip them and reconstitute a single CoerceToDomain over
* the combined FieldStore/SubscriptingRef nodes. (Notice that this has the
* result that the domain's checks are applied only after we do all the
* field or element updates, not after each one. This is arguably desirable.)
*----------
*/
src_expr = (Node *) src_tle->expr;
prior_expr = (Node *) prior_tle->expr;
if (src_expr && IsA(src_expr, CoerceToDomain) &&
prior_expr && IsA(prior_expr, CoerceToDomain) &&
((CoerceToDomain *) src_expr)->resulttype ==
((CoerceToDomain *) prior_expr)->resulttype)
{
/* we assume without checking that resulttypmod/resultcollid match */
coerce_expr = (CoerceToDomain *) src_expr;
src_expr = (Node *) ((CoerceToDomain *) src_expr)->arg;
prior_expr = (Node *) ((CoerceToDomain *) prior_expr)->arg;
}
src_input = get_assignment_input(src_expr);
prior_input = get_assignment_input(prior_expr);
if (src_input == NULL ||
prior_input == NULL ||
exprType(src_expr) != exprType(prior_expr))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("multiple assignments to same column \"%s\"",
attrName)));
/*
* Prior TLE could be a nest of assignments if we do this more than once.
*/
priorbottom = prior_input;
for (;;)
{
Node *newbottom = get_assignment_input(priorbottom);
if (newbottom == NULL)
break; /* found the original Var reference */
priorbottom = newbottom;
}
if (!equal(priorbottom, src_input))
ereport(ERROR,
(errcode(ERRCODE_SYNTAX_ERROR),
errmsg("multiple assignments to same column \"%s\"",
attrName)));
/*
* Looks OK to nest 'em.
*/
if (IsA(src_expr, FieldStore))
{
FieldStore *fstore = makeNode(FieldStore);
if (IsA(prior_expr, FieldStore))
{
/* combine the two */
memcpy(fstore, prior_expr, sizeof(FieldStore));
fstore->newvals =
list_concat_copy(((FieldStore *) prior_expr)->newvals,
((FieldStore *) src_expr)->newvals);
fstore->fieldnums =
list_concat_copy(((FieldStore *) prior_expr)->fieldnums,
((FieldStore *) src_expr)->fieldnums);
}
else
{
/* general case, just nest 'em */
memcpy(fstore, src_expr, sizeof(FieldStore));
fstore->arg = (Expr *) prior_expr;
}
newexpr = (Node *) fstore;
}
else if (IsA(src_expr, SubscriptingRef))
{
SubscriptingRef *sbsref = makeNode(SubscriptingRef);
memcpy(sbsref, src_expr, sizeof(SubscriptingRef));
sbsref->refexpr = (Expr *) prior_expr;
newexpr = (Node *) sbsref;
}
else
{
elog(ERROR, "cannot happen");
newexpr = NULL;
}
if (coerce_expr)
{
/* put back the CoerceToDomain */
CoerceToDomain *newcoerce = makeNode(CoerceToDomain);
memcpy(newcoerce, coerce_expr, sizeof(CoerceToDomain));
newcoerce->arg = (Expr *) newexpr;
newexpr = (Node *) newcoerce;
}
result = flatCopyTargetEntry(src_tle);
result->expr = (Expr *) newexpr;
return result;
}
/*
* If node is an assignment node, return its input; else return NULL
*/
static Node *
get_assignment_input(Node *node)
{
if (node == NULL)
return NULL;
if (IsA(node, FieldStore))
{
FieldStore *fstore = (FieldStore *) node;
return (Node *) fstore->arg;
}
else if (IsA(node, SubscriptingRef))
{
SubscriptingRef *sbsref = (SubscriptingRef *) node;
if (sbsref->refassgnexpr == NULL)
return NULL;
return (Node *) sbsref->refexpr;
}
return NULL;
}
/*
* Make an expression tree for the default value for a column.
*
* If there is no default, return a NULL instead.
*/
Node *
build_column_default(Relation rel, int attrno)
{
TupleDesc rd_att = rel->rd_att;
Form_pg_attribute att_tup = TupleDescAttr(rd_att, attrno - 1);
Oid atttype = att_tup->atttypid;
int32 atttypmod = att_tup->atttypmod;
Node *expr = NULL;
Oid exprtype;
if (att_tup->attidentity)
{
NextValueExpr *nve = makeNode(NextValueExpr);
nve->seqid = getIdentitySequence(RelationGetRelid(rel), attrno, false);
nve->typeId = att_tup->atttypid;
return (Node *) nve;
}
/*
Clean up treatment of missing default and CHECK-constraint records. Andrew Gierth reported that it's possible to crash the backend if no pg_attrdef record is found to match an attribute that has atthasdef set. AttrDefaultFetch warns about this situation, but then leaves behind a relation tupdesc that has null "adbin" pointer(s), which most places don't guard against. We considered promoting the warning to an error, but throwing errors during relcache load is pretty drastic: it effectively locks one out of using the relation at all. What seems better is to leave the load-time behavior as a warning, but then throw an error in any code path that wants to use a default and can't find it. This confines the error to a subset of INSERT/UPDATE operations on the table, and in particular will at least allow a pg_dump to succeed. Also, we should fix AttrDefaultFetch to not leave any null pointers in the tupdesc, because that just creates an untested bug hazard. While at it, apply the same philosophy of "warn at load, throw error only upon use of the known-missing info" to CHECK constraints. CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch, but for reasons lost in the mists of time, it was throwing ERROR for the same cases that AttrDefaultFetch treats as WARNING. Make the two functions more nearly alike. In passing, get rid of potentially-O(N^2) loops in equalTupleDesc by making AttrDefaultFetch sort the entries after fetching them, so that equalTupleDesc can assume that entries in two equal tupdescs must be in matching order. (CheckConstraintFetch already was sorting CHECK constraints, but equalTupleDesc hadn't been told about it.) There's some argument for back-patching this, but with such a small number of field reports, I'm content to fix it in HEAD. Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
2021-04-06 16:34:37 +02:00
* If relation has a default for this column, fetch that expression.
*/
Clean up treatment of missing default and CHECK-constraint records. Andrew Gierth reported that it's possible to crash the backend if no pg_attrdef record is found to match an attribute that has atthasdef set. AttrDefaultFetch warns about this situation, but then leaves behind a relation tupdesc that has null "adbin" pointer(s), which most places don't guard against. We considered promoting the warning to an error, but throwing errors during relcache load is pretty drastic: it effectively locks one out of using the relation at all. What seems better is to leave the load-time behavior as a warning, but then throw an error in any code path that wants to use a default and can't find it. This confines the error to a subset of INSERT/UPDATE operations on the table, and in particular will at least allow a pg_dump to succeed. Also, we should fix AttrDefaultFetch to not leave any null pointers in the tupdesc, because that just creates an untested bug hazard. While at it, apply the same philosophy of "warn at load, throw error only upon use of the known-missing info" to CHECK constraints. CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch, but for reasons lost in the mists of time, it was throwing ERROR for the same cases that AttrDefaultFetch treats as WARNING. Make the two functions more nearly alike. In passing, get rid of potentially-O(N^2) loops in equalTupleDesc by making AttrDefaultFetch sort the entries after fetching them, so that equalTupleDesc can assume that entries in two equal tupdescs must be in matching order. (CheckConstraintFetch already was sorting CHECK constraints, but equalTupleDesc hadn't been told about it.) There's some argument for back-patching this, but with such a small number of field reports, I'm content to fix it in HEAD. Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
2021-04-06 16:34:37 +02:00
if (att_tup->atthasdef)
{
Clean up treatment of missing default and CHECK-constraint records. Andrew Gierth reported that it's possible to crash the backend if no pg_attrdef record is found to match an attribute that has atthasdef set. AttrDefaultFetch warns about this situation, but then leaves behind a relation tupdesc that has null "adbin" pointer(s), which most places don't guard against. We considered promoting the warning to an error, but throwing errors during relcache load is pretty drastic: it effectively locks one out of using the relation at all. What seems better is to leave the load-time behavior as a warning, but then throw an error in any code path that wants to use a default and can't find it. This confines the error to a subset of INSERT/UPDATE operations on the table, and in particular will at least allow a pg_dump to succeed. Also, we should fix AttrDefaultFetch to not leave any null pointers in the tupdesc, because that just creates an untested bug hazard. While at it, apply the same philosophy of "warn at load, throw error only upon use of the known-missing info" to CHECK constraints. CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch, but for reasons lost in the mists of time, it was throwing ERROR for the same cases that AttrDefaultFetch treats as WARNING. Make the two functions more nearly alike. In passing, get rid of potentially-O(N^2) loops in equalTupleDesc by making AttrDefaultFetch sort the entries after fetching them, so that equalTupleDesc can assume that entries in two equal tupdescs must be in matching order. (CheckConstraintFetch already was sorting CHECK constraints, but equalTupleDesc hadn't been told about it.) There's some argument for back-patching this, but with such a small number of field reports, I'm content to fix it in HEAD. Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
2021-04-06 16:34:37 +02:00
if (rd_att->constr && rd_att->constr->num_defval > 0)
{
Clean up treatment of missing default and CHECK-constraint records. Andrew Gierth reported that it's possible to crash the backend if no pg_attrdef record is found to match an attribute that has atthasdef set. AttrDefaultFetch warns about this situation, but then leaves behind a relation tupdesc that has null "adbin" pointer(s), which most places don't guard against. We considered promoting the warning to an error, but throwing errors during relcache load is pretty drastic: it effectively locks one out of using the relation at all. What seems better is to leave the load-time behavior as a warning, but then throw an error in any code path that wants to use a default and can't find it. This confines the error to a subset of INSERT/UPDATE operations on the table, and in particular will at least allow a pg_dump to succeed. Also, we should fix AttrDefaultFetch to not leave any null pointers in the tupdesc, because that just creates an untested bug hazard. While at it, apply the same philosophy of "warn at load, throw error only upon use of the known-missing info" to CHECK constraints. CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch, but for reasons lost in the mists of time, it was throwing ERROR for the same cases that AttrDefaultFetch treats as WARNING. Make the two functions more nearly alike. In passing, get rid of potentially-O(N^2) loops in equalTupleDesc by making AttrDefaultFetch sort the entries after fetching them, so that equalTupleDesc can assume that entries in two equal tupdescs must be in matching order. (CheckConstraintFetch already was sorting CHECK constraints, but equalTupleDesc hadn't been told about it.) There's some argument for back-patching this, but with such a small number of field reports, I'm content to fix it in HEAD. Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
2021-04-06 16:34:37 +02:00
AttrDefault *defval = rd_att->constr->defval;
int ndef = rd_att->constr->num_defval;
while (--ndef >= 0)
{
Clean up treatment of missing default and CHECK-constraint records. Andrew Gierth reported that it's possible to crash the backend if no pg_attrdef record is found to match an attribute that has atthasdef set. AttrDefaultFetch warns about this situation, but then leaves behind a relation tupdesc that has null "adbin" pointer(s), which most places don't guard against. We considered promoting the warning to an error, but throwing errors during relcache load is pretty drastic: it effectively locks one out of using the relation at all. What seems better is to leave the load-time behavior as a warning, but then throw an error in any code path that wants to use a default and can't find it. This confines the error to a subset of INSERT/UPDATE operations on the table, and in particular will at least allow a pg_dump to succeed. Also, we should fix AttrDefaultFetch to not leave any null pointers in the tupdesc, because that just creates an untested bug hazard. While at it, apply the same philosophy of "warn at load, throw error only upon use of the known-missing info" to CHECK constraints. CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch, but for reasons lost in the mists of time, it was throwing ERROR for the same cases that AttrDefaultFetch treats as WARNING. Make the two functions more nearly alike. In passing, get rid of potentially-O(N^2) loops in equalTupleDesc by making AttrDefaultFetch sort the entries after fetching them, so that equalTupleDesc can assume that entries in two equal tupdescs must be in matching order. (CheckConstraintFetch already was sorting CHECK constraints, but equalTupleDesc hadn't been told about it.) There's some argument for back-patching this, but with such a small number of field reports, I'm content to fix it in HEAD. Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
2021-04-06 16:34:37 +02:00
if (attrno == defval[ndef].adnum)
{
/* Found it, convert string representation to node tree. */
expr = stringToNode(defval[ndef].adbin);
break;
}
}
}
Clean up treatment of missing default and CHECK-constraint records. Andrew Gierth reported that it's possible to crash the backend if no pg_attrdef record is found to match an attribute that has atthasdef set. AttrDefaultFetch warns about this situation, but then leaves behind a relation tupdesc that has null "adbin" pointer(s), which most places don't guard against. We considered promoting the warning to an error, but throwing errors during relcache load is pretty drastic: it effectively locks one out of using the relation at all. What seems better is to leave the load-time behavior as a warning, but then throw an error in any code path that wants to use a default and can't find it. This confines the error to a subset of INSERT/UPDATE operations on the table, and in particular will at least allow a pg_dump to succeed. Also, we should fix AttrDefaultFetch to not leave any null pointers in the tupdesc, because that just creates an untested bug hazard. While at it, apply the same philosophy of "warn at load, throw error only upon use of the known-missing info" to CHECK constraints. CheckConstraintFetch is very nearly the same logic as AttrDefaultFetch, but for reasons lost in the mists of time, it was throwing ERROR for the same cases that AttrDefaultFetch treats as WARNING. Make the two functions more nearly alike. In passing, get rid of potentially-O(N^2) loops in equalTupleDesc by making AttrDefaultFetch sort the entries after fetching them, so that equalTupleDesc can assume that entries in two equal tupdescs must be in matching order. (CheckConstraintFetch already was sorting CHECK constraints, but equalTupleDesc hadn't been told about it.) There's some argument for back-patching this, but with such a small number of field reports, I'm content to fix it in HEAD. Discussion: https://postgr.es/m/87pmzaq4gx.fsf@news-spur.riddles.org.uk
2021-04-06 16:34:37 +02:00
if (expr == NULL)
elog(ERROR, "default expression not found for attribute %d of relation \"%s\"",
attrno, RelationGetRelationName(rel));
}
/*
* No per-column default, so look for a default for the type itself. But
* not for generated columns.
*/
if (expr == NULL && !att_tup->attgenerated)
expr = get_typdefault(atttype);
if (expr == NULL)
return NULL; /* No default anywhere */
/*
* Make sure the value is coerced to the target column type; this will
* generally be true already, but there seem to be some corner cases
* involving domain defaults where it might not be true. This should match
* the parser's processing of non-defaulted expressions --- see
* transformAssignedExpr().
*/
exprtype = exprType(expr);
expr = coerce_to_target_type(NULL, /* no UNKNOWN params here */
expr, exprtype,
atttype, atttypmod,
COERCION_ASSIGNMENT,
COERCE_IMPLICIT_CAST,
-1);
if (expr == NULL)
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" is of type %s"
" but default expression is of type %s",
NameStr(att_tup->attname),
format_type_be(atttype),
format_type_be(exprtype)),
errhint("You will need to rewrite or cast the expression.")));
return expr;
}
/* Does VALUES RTE contain any SetToDefault items? */
static bool
searchForDefault(RangeTblEntry *rte)
{
ListCell *lc;
foreach(lc, rte->values_lists)
{
List *sublist = (List *) lfirst(lc);
ListCell *lc2;
foreach(lc2, sublist)
{
Node *col = (Node *) lfirst(lc2);
if (IsA(col, SetToDefault))
return true;
}
}
return false;
}
/*
* Search a VALUES RTE for columns that contain only SetToDefault items,
* returning a Bitmapset containing the attribute numbers of any such columns.
*/
static Bitmapset *
findDefaultOnlyColumns(RangeTblEntry *rte)
{
Bitmapset *default_only_cols = NULL;
ListCell *lc;
foreach(lc, rte->values_lists)
{
List *sublist = (List *) lfirst(lc);
ListCell *lc2;
int i;
if (default_only_cols == NULL)
{
/* Populate the initial result bitmap from the first row */
i = 0;
foreach(lc2, sublist)
{
Node *col = (Node *) lfirst(lc2);
i++;
if (IsA(col, SetToDefault))
default_only_cols = bms_add_member(default_only_cols, i);
}
}
else
{
/* Update the result bitmap from this next row */
i = 0;
foreach(lc2, sublist)
{
Node *col = (Node *) lfirst(lc2);
i++;
if (!IsA(col, SetToDefault))
default_only_cols = bms_del_member(default_only_cols, i);
}
}
/*
* If no column in the rows read so far contains only DEFAULT items,
* we are done.
*/
if (bms_is_empty(default_only_cols))
break;
}
return default_only_cols;
}
/*
* When processing INSERT ... VALUES with a VALUES RTE (ie, multiple VALUES
* lists), we have to replace any DEFAULT items in the VALUES lists with
* the appropriate default expressions. The other aspects of targetlist
* rewriting need be applied only to the query's targetlist proper.
*
* For an auto-updatable view, each DEFAULT item in the VALUES list is
* replaced with the default from the view, if it has one. Otherwise it is
* left untouched so that the underlying base relation's default can be
* applied instead (when we later recurse to here after rewriting the query
* to refer to the base relation instead of the view).
*
* For other types of relation, including rule- and trigger-updatable views,
* all DEFAULT items are replaced, and if the target relation doesn't have a
* default, the value is explicitly set to NULL.
*
* Also, if a DEFAULT item is found in a column mentioned in unused_cols,
* it is explicitly set to NULL. This happens for columns in the VALUES RTE
* whose corresponding targetlist entries have already been replaced with the
* relation's default expressions, so that any values in those columns of the
* VALUES RTE are no longer used. This can happen for identity and generated
* columns (if INSERT ... OVERRIDING USER VALUE is used, or all the values to
* be inserted are DEFAULT). In principle we could replace all entries in
* such a column with NULL, whether DEFAULT or not; but it doesn't seem worth
* the trouble.
*
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
* Note that we may have subscripted or field assignment targetlist entries,
* as well as more complex expressions from already-replaced DEFAULT items if
* we have recursed to here for an auto-updatable view. However, it ought to
* be impossible for such entries to have DEFAULTs assigned to them, except
* for unused columns, as described above --- we should only have to replace
* DEFAULT items for targetlist entries that contain simple Vars referencing
* the VALUES RTE, or which are no longer referred to by the targetlist.
*
* Returns true if all DEFAULT items were replaced, and false if some were
* left untouched.
*/
static bool
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
rewriteValuesRTE(Query *parsetree, RangeTblEntry *rte, int rti,
Relation target_relation,
Bitmapset *unused_cols)
{
List *newValues;
ListCell *lc;
bool isAutoUpdatableView;
bool allReplaced;
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
int numattrs;
int *attrnos;
/* Steps below are not sensible for non-INSERT queries */
Assert(parsetree->commandType == CMD_INSERT);
Assert(rte->rtekind == RTE_VALUES);
/*
* Rebuilding all the lists is a pretty expensive proposition in a big
* VALUES list, and it's a waste of time if there aren't any DEFAULT
* placeholders. So first scan to see if there are any.
*/
if (!searchForDefault(rte))
return true; /* nothing to do */
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
/*
* Scan the targetlist for entries referring to the VALUES RTE, and note
* the target attributes. As noted above, we should only need to do this
* for targetlist entries containing simple Vars --- nothing else in the
* VALUES RTE should contain DEFAULT items (except possibly for unused
* columns), and we complain if such a thing does occur.
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
*/
numattrs = list_length(linitial(rte->values_lists));
attrnos = (int *) palloc0(numattrs * sizeof(int));
foreach(lc, parsetree->targetList)
{
TargetEntry *tle = (TargetEntry *) lfirst(lc);
if (IsA(tle->expr, Var))
{
Var *var = (Var *) tle->expr;
if (var->varno == rti)
{
int attrno = var->varattno;
Assert(attrno >= 1 && attrno <= numattrs);
attrnos[attrno - 1] = tle->resno;
}
}
}
/*
* Check if the target relation is an auto-updatable view, in which case
* unresolved defaults will be left untouched rather than being set to
* NULL.
*/
isAutoUpdatableView = false;
if (target_relation->rd_rel->relkind == RELKIND_VIEW &&
!view_has_instead_trigger(target_relation, CMD_INSERT))
{
List *locks;
bool hasUpdate;
bool found;
ListCell *l;
/* Look for an unconditional DO INSTEAD rule */
locks = matchLocks(CMD_INSERT, target_relation->rd_rules,
parsetree->resultRelation, parsetree, &hasUpdate);
found = false;
foreach(l, locks)
{
RewriteRule *rule_lock = (RewriteRule *) lfirst(l);
if (rule_lock->isInstead &&
rule_lock->qual == NULL)
{
found = true;
break;
}
}
/*
* If we didn't find an unconditional DO INSTEAD rule, assume that the
* view is auto-updatable. If it isn't, rewriteTargetView() will
* throw an error.
*/
if (!found)
isAutoUpdatableView = true;
}
newValues = NIL;
allReplaced = true;
foreach(lc, rte->values_lists)
{
List *sublist = (List *) lfirst(lc);
List *newList = NIL;
ListCell *lc2;
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
int i;
Assert(list_length(sublist) == numattrs);
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
i = 0;
foreach(lc2, sublist)
{
Node *col = (Node *) lfirst(lc2);
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
int attrno = attrnos[i++];
if (IsA(col, SetToDefault))
{
Form_pg_attribute att_tup;
Node *new_expr;
/*
* If this column isn't used, just replace the DEFAULT with
* NULL (attrno will be 0 in this case because the targetlist
* entry will have been replaced by the default expression).
*/
if (bms_is_member(i, unused_cols))
{
SetToDefault *def = (SetToDefault *) col;
newList = lappend(newList,
makeNullConst(def->typeId,
def->typeMod,
def->collation));
continue;
}
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
if (attrno == 0)
elog(ERROR, "cannot set value in column %d to DEFAULT", i);
Assert(attrno > 0 && attrno <= target_relation->rd_att->natts);
att_tup = TupleDescAttr(target_relation->rd_att, attrno - 1);
if (!att_tup->attisdropped)
new_expr = build_column_default(target_relation, attrno);
else
new_expr = NULL; /* force a NULL if dropped */
/*
* If there is no default (ie, default is effectively NULL),
* we've got to explicitly set the column to NULL, unless the
* target relation is an auto-updatable view.
*/
if (!new_expr)
{
if (isAutoUpdatableView)
{
/* Leave the value untouched */
newList = lappend(newList, col);
allReplaced = false;
continue;
}
new_expr = (Node *) makeConst(att_tup->atttypid,
-1,
att_tup->attcollation,
att_tup->attlen,
(Datum) 0,
true, /* isnull */
att_tup->attbyval);
/* this is to catch a NOT NULL domain constraint */
new_expr = coerce_to_domain(new_expr,
InvalidOid, -1,
att_tup->atttypid,
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
COERCION_IMPLICIT,
COERCE_IMPLICIT_CAST,
-1,
false);
}
newList = lappend(newList, new_expr);
}
else
newList = lappend(newList, col);
}
newValues = lappend(newValues, newList);
}
rte->values_lists = newValues;
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
pfree(attrnos);
return allReplaced;
}
/*
* Mop up any remaining DEFAULT items in the given VALUES RTE by
* replacing them with NULL constants.
*
* This is used for the product queries generated by DO ALSO rules attached to
* an auto-updatable view. The action can't depend on the "target relation"
* since the product query might not have one (it needn't be an INSERT).
* Essentially, such queries are treated as being attached to a rule-updatable
* view.
*/
static void
rewriteValuesRTEToNulls(Query *parsetree, RangeTblEntry *rte)
{
List *newValues;
ListCell *lc;
newValues = NIL;
foreach(lc, rte->values_lists)
{
List *sublist = (List *) lfirst(lc);
List *newList = NIL;
ListCell *lc2;
foreach(lc2, sublist)
{
Node *col = (Node *) lfirst(lc2);
if (IsA(col, SetToDefault))
{
SetToDefault *def = (SetToDefault *) col;
newList = lappend(newList, makeNullConst(def->typeId,
def->typeMod,
def->collation));
}
else
newList = lappend(newList, col);
}
newValues = lappend(newValues, newList);
}
rte->values_lists = newValues;
}
/*
* matchLocks -
* match the list of locks and returns the matching rules
*/
static List *
matchLocks(CmdType event,
RuleLock *rulelocks,
int varno,
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
Query *parsetree,
bool *hasUpdate)
{
List *matching_locks = NIL;
int nlocks;
int i;
if (rulelocks == NULL)
return NIL;
if (parsetree->commandType != CMD_SELECT)
{
if (parsetree->resultRelation != varno)
return NIL;
}
nlocks = rulelocks->numLocks;
for (i = 0; i < nlocks; i++)
{
RewriteRule *oneLock = rulelocks->rules[i];
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
if (oneLock->event == CMD_UPDATE)
*hasUpdate = true;
Changes pg_trigger and extend pg_rewrite in order to allow triggers and rules to be defined with different, per session controllable, behaviors for replication purposes. This will allow replication systems like Slony-I and, as has been stated on pgsql-hackers, other products to control the firing mechanism of triggers and rewrite rules without modifying the system catalog directly. The firing mechanisms are controlled by a new superuser-only GUC variable, session_replication_role, together with a change to pg_trigger.tgenabled and a new column pg_rewrite.ev_enabled. Both columns are a single char data type now (tgenabled was a bool before). The possible values in these attributes are: 'O' - Trigger/Rule fires when session_replication_role is "origin" (default) or "local". This is the default behavior. 'D' - Trigger/Rule is disabled and fires never 'A' - Trigger/Rule fires always regardless of the setting of session_replication_role 'R' - Trigger/Rule fires when session_replication_role is "replica" The GUC variable can only be changed as long as the system does not have any cached query plans. This will prevent changing the session role and accidentally executing stored procedures or functions that have plans cached that expand to the wrong query set due to differences in the rule firing semantics. The SQL syntax for changing a triggers/rules firing semantics is ALTER TABLE <tabname> <when> TRIGGER|RULE <name>; <when> ::= ENABLE | ENABLE ALWAYS | ENABLE REPLICA | DISABLE psql's \d command as well as pg_dump are extended in a backward compatible fashion. Jan
2007-03-20 00:38:32 +01:00
/*
* Suppress ON INSERT/UPDATE/DELETE rules that are disabled or
* configured to not fire during the current sessions replication
* role. ON SELECT rules will always be applied in order to keep views
* working even in LOCAL or REPLICA role.
Changes pg_trigger and extend pg_rewrite in order to allow triggers and rules to be defined with different, per session controllable, behaviors for replication purposes. This will allow replication systems like Slony-I and, as has been stated on pgsql-hackers, other products to control the firing mechanism of triggers and rewrite rules without modifying the system catalog directly. The firing mechanisms are controlled by a new superuser-only GUC variable, session_replication_role, together with a change to pg_trigger.tgenabled and a new column pg_rewrite.ev_enabled. Both columns are a single char data type now (tgenabled was a bool before). The possible values in these attributes are: 'O' - Trigger/Rule fires when session_replication_role is "origin" (default) or "local". This is the default behavior. 'D' - Trigger/Rule is disabled and fires never 'A' - Trigger/Rule fires always regardless of the setting of session_replication_role 'R' - Trigger/Rule fires when session_replication_role is "replica" The GUC variable can only be changed as long as the system does not have any cached query plans. This will prevent changing the session role and accidentally executing stored procedures or functions that have plans cached that expand to the wrong query set due to differences in the rule firing semantics. The SQL syntax for changing a triggers/rules firing semantics is ALTER TABLE <tabname> <when> TRIGGER|RULE <name>; <when> ::= ENABLE | ENABLE ALWAYS | ENABLE REPLICA | DISABLE psql's \d command as well as pg_dump are extended in a backward compatible fashion. Jan
2007-03-20 00:38:32 +01:00
*/
if (oneLock->event != CMD_SELECT)
{
if (SessionReplicationRole == SESSION_REPLICATION_ROLE_REPLICA)
{
if (oneLock->enabled == RULE_FIRES_ON_ORIGIN ||
oneLock->enabled == RULE_DISABLED)
continue;
}
2007-11-16 02:51:22 +01:00
else /* ORIGIN or LOCAL ROLE */
Changes pg_trigger and extend pg_rewrite in order to allow triggers and rules to be defined with different, per session controllable, behaviors for replication purposes. This will allow replication systems like Slony-I and, as has been stated on pgsql-hackers, other products to control the firing mechanism of triggers and rewrite rules without modifying the system catalog directly. The firing mechanisms are controlled by a new superuser-only GUC variable, session_replication_role, together with a change to pg_trigger.tgenabled and a new column pg_rewrite.ev_enabled. Both columns are a single char data type now (tgenabled was a bool before). The possible values in these attributes are: 'O' - Trigger/Rule fires when session_replication_role is "origin" (default) or "local". This is the default behavior. 'D' - Trigger/Rule is disabled and fires never 'A' - Trigger/Rule fires always regardless of the setting of session_replication_role 'R' - Trigger/Rule fires when session_replication_role is "replica" The GUC variable can only be changed as long as the system does not have any cached query plans. This will prevent changing the session role and accidentally executing stored procedures or functions that have plans cached that expand to the wrong query set due to differences in the rule firing semantics. The SQL syntax for changing a triggers/rules firing semantics is ALTER TABLE <tabname> <when> TRIGGER|RULE <name>; <when> ::= ENABLE | ENABLE ALWAYS | ENABLE REPLICA | DISABLE psql's \d command as well as pg_dump are extended in a backward compatible fashion. Jan
2007-03-20 00:38:32 +01:00
{
if (oneLock->enabled == RULE_FIRES_ON_REPLICA ||
oneLock->enabled == RULE_DISABLED)
continue;
}
}
if (oneLock->event == event)
{
if (parsetree->commandType != CMD_SELECT ||
rangeTableEntry_used((Node *) parsetree, varno, 0))
matching_locks = lappend(matching_locks, oneLock);
}
}
return matching_locks;
}
/*
* ApplyRetrieveRule - expand an ON SELECT rule
*/
static Query *
ApplyRetrieveRule(Query *parsetree,
RewriteRule *rule,
int rt_index,
Relation relation,
List *activeRIRs)
{
Query *rule_action;
RangeTblEntry *rte,
*subrte;
RowMarkClause *rc;
int numCols;
if (list_length(rule->actions) != 1)
elog(ERROR, "expected just one rule action");
if (rule->qual != NULL)
elog(ERROR, "cannot handle qualified ON SELECT rule");
if (rt_index == parsetree->resultRelation)
{
/*
* We have a view as the result relation of the query, and it wasn't
* rewritten by any rule. This case is supported if there is an
* INSTEAD OF trigger that will trap attempts to insert/update/delete
* view rows. The executor will check that; for the moment just plow
* ahead. We have two cases:
*
* For INSERT, we needn't do anything. The unmodified RTE will serve
* fine as the result relation.
*
* For UPDATE/DELETE, we need to expand the view so as to have source
* data for the operation. But we also need an unmodified RTE to
* serve as the target. So, copy the RTE and add the copy to the
* rangetable. Note that the copy does not get added to the jointree.
* Also note that there's a hack in fireRIRrules to avoid calling this
* function again when it arrives at the copied RTE.
*/
if (parsetree->commandType == CMD_INSERT)
return parsetree;
else if (parsetree->commandType == CMD_UPDATE ||
parsetree->commandType == CMD_DELETE)
{
RangeTblEntry *newrte;
Fix creation of resjunk tlist entries for inherited mixed UPDATE/DELETE. rewriteTargetListUD's processing is dependent on the relkind of the query's target table. That was fine at the time it was made to act that way, even for queries on inheritance trees, because all tables in an inheritance tree would necessarily be plain tables. However, the 9.5 feature addition allowing some members of an inheritance tree to be foreign tables broke the assumption that rewriteTargetListUD's output tlist could be applied to all child tables with nothing more than column-number mapping. This led to visible failures if foreign child tables had row-level triggers, and would also break in cases where child tables belonged to FDWs that used methods other than CTID for row identification. To fix, delay running rewriteTargetListUD until after the planner has expanded inheritance, so that it is applied separately to the (already mapped) tlist for each child table. We can conveniently call it from preprocess_targetlist. Refactor associated code slightly to avoid the need to heap_open the target relation multiple times during preprocess_targetlist. (The APIs remain a bit ugly, particularly around the point of which steps scribble on parse->targetList and which don't. But avoiding such scribbling would require a change in FDW callback APIs, which is more pain than it's worth.) Also fix ExecModifyTable to ensure that "tupleid" is reset to NULL when we transition from rows providing a CTID to rows that don't. (That's really an independent bug, but it manifests in much the same cases.) Add a regression test checking one manifestation of this problem, which was that row-level triggers on a foreign child table did not work right. Back-patch to 9.5 where the problem was introduced. Etsuro Fujita, reviewed by Ildus Kurbangaliev and Ashutosh Bapat Discussion: https://postgr.es/m/20170514150525.0346ba72@postgrespro.ru
2017-11-27 23:53:56 +01:00
Var *var;
TargetEntry *tle;
rte = rt_fetch(rt_index, parsetree->rtable);
newrte = copyObject(rte);
parsetree->rtable = lappend(parsetree->rtable, newrte);
parsetree->resultRelation = list_length(parsetree->rtable);
/*
* There's no need to do permissions checks twice, so wipe out the
* permissions info for the original RTE (we prefer to keep the
* bits set on the result RTE).
*/
rte->requiredPerms = 0;
rte->checkAsUser = InvalidOid;
rte->selectedCols = NULL;
rte->insertedCols = NULL;
rte->updatedCols = NULL;
Calculate extraUpdatedCols in query rewriter, not parser. It's unsafe to do this at parse time because addition of generated columns to a table would not invalidate stored rules containing UPDATEs on the table ... but there might now be dependent generated columns that were not there when the rule was made. This also fixes an oversight that rewriteTargetView failed to update extraUpdatedCols when transforming an UPDATE on an updatable view. (Since the new calculation is downstream of that, rewriteTargetView doesn't actually need to do anything; but before, there was a demonstrable bug there.) In v13 and HEAD, this leads to easily-visible bugs because (since commit c6679e4fc) we won't recalculate generated columns that aren't listed in extraUpdatedCols. In v12 this bitmap is mostly just used for trigger-firing decisions, so you'd only notice a problem if a trigger cared whether a generated column had been updated. I'd complained about this back in May, but then forgot about it until bug #16671 from Michael Paul Killian revived the issue. Back-patch to v12 where this field was introduced. If existing stored rules contain any extraUpdatedCols values, they'll be ignored because the rewriter will overwrite them, so the bug will be fixed even for existing rules. (But note that if someone were to update to 13.1 or 12.5, store some rules with UPDATEs on tables having generated columns, and then downgrade to a prior minor version, they might observe issues similar to what this patch fixes. That seems unlikely enough to not be worth going to a lot of effort to fix.) Discussion: https://postgr.es/m/10206.1588964727@sss.pgh.pa.us Discussion: https://postgr.es/m/16671-2fa55851859fb166@postgresql.org
2020-10-28 18:47:02 +01:00
rte->extraUpdatedCols = NULL;
/*
* For the most part, Vars referencing the view should remain as
* they are, meaning that they implicitly represent OLD values.
* But in the RETURNING list if any, we want such Vars to
* represent NEW values, so change them to reference the new RTE.
*
* Since ChangeVarNodes scribbles on the tree in-place, copy the
* RETURNING list first for safety.
*/
parsetree->returningList = copyObject(parsetree->returningList);
ChangeVarNodes((Node *) parsetree->returningList, rt_index,
parsetree->resultRelation, 0);
Fix creation of resjunk tlist entries for inherited mixed UPDATE/DELETE. rewriteTargetListUD's processing is dependent on the relkind of the query's target table. That was fine at the time it was made to act that way, even for queries on inheritance trees, because all tables in an inheritance tree would necessarily be plain tables. However, the 9.5 feature addition allowing some members of an inheritance tree to be foreign tables broke the assumption that rewriteTargetListUD's output tlist could be applied to all child tables with nothing more than column-number mapping. This led to visible failures if foreign child tables had row-level triggers, and would also break in cases where child tables belonged to FDWs that used methods other than CTID for row identification. To fix, delay running rewriteTargetListUD until after the planner has expanded inheritance, so that it is applied separately to the (already mapped) tlist for each child table. We can conveniently call it from preprocess_targetlist. Refactor associated code slightly to avoid the need to heap_open the target relation multiple times during preprocess_targetlist. (The APIs remain a bit ugly, particularly around the point of which steps scribble on parse->targetList and which don't. But avoiding such scribbling would require a change in FDW callback APIs, which is more pain than it's worth.) Also fix ExecModifyTable to ensure that "tupleid" is reset to NULL when we transition from rows providing a CTID to rows that don't. (That's really an independent bug, but it manifests in much the same cases.) Add a regression test checking one manifestation of this problem, which was that row-level triggers on a foreign child table did not work right. Back-patch to 9.5 where the problem was introduced. Etsuro Fujita, reviewed by Ildus Kurbangaliev and Ashutosh Bapat Discussion: https://postgr.es/m/20170514150525.0346ba72@postgrespro.ru
2017-11-27 23:53:56 +01:00
/*
* To allow the executor to compute the original view row to pass
* to the INSTEAD OF trigger, we add a resjunk whole-row Var
* referencing the original RTE. This will later get expanded
* into a RowExpr computing all the OLD values of the view row.
*/
var = makeWholeRowVar(rte, rt_index, 0, false);
tle = makeTargetEntry((Expr *) var,
list_length(parsetree->targetList) + 1,
pstrdup("wholerow"),
true);
parsetree->targetList = lappend(parsetree->targetList, tle);
/* Now, continue with expanding the original view RTE */
}
else
elog(ERROR, "unrecognized commandType: %d",
(int) parsetree->commandType);
}
/*
* Check if there's a FOR [KEY] UPDATE/SHARE clause applying to this view.
*
* Note: we needn't explicitly consider any such clauses appearing in
* ancestor query levels; their effects have already been pushed down to
* here by markQueryForLocking, and will be reflected in "rc".
*/
rc = get_parse_rowmark(parsetree, rt_index);
/*
* Make a modifiable copy of the view query, and acquire needed locks on
* the relations it mentions. Force at least RowShareLock for all such
* rels if there's a FOR [KEY] UPDATE/SHARE clause affecting this view.
*/
rule_action = copyObject(linitial(rule->actions));
AcquireRewriteLocks(rule_action, true, (rc != NULL));
/*
* If FOR [KEY] UPDATE/SHARE of view, mark all the contained tables as
* implicit FOR [KEY] UPDATE/SHARE, the same as the parser would have done
* if the view's subquery had been written out explicitly.
*/
if (rc != NULL)
markQueryForLocking(rule_action, (Node *) rule_action->jointree,
rc->strength, rc->waitPolicy, true);
/*
* Recursively expand any view references inside the view.
*
* Note: this must happen after markQueryForLocking. That way, any UPDATE
* permission bits needed for sub-views are initially applied to their
* RTE_RELATION RTEs by markQueryForLocking, and then transferred to their
* OLD rangetable entries by the action below (in a recursive call of this
* routine).
*/
rule_action = fireRIRrules(rule_action, activeRIRs);
/*
Fix some minor issues exposed by outfuncs/readfuncs testing. A test patch to pass parse and plan trees through outfuncs + readfuncs exposed several issues that need to be fixed to get clean matches: Query.withCheckOptions failed to get copied; it's intentionally ignored by outfuncs/readfuncs on the grounds that it'd always be NIL anyway in stored rules. This seems less than future-proof, and it's not even saving very much, so just undo the decision and treat the field like all others. Several places that convert a view RTE into a subquery RTE, or similar manipulations, failed to clear out fields that were specific to the original RTE type and should be zero in a subquery RTE. Since readfuncs.c will leave such fields as zero, equalfuncs.c thinks the nodes are different leading to a reported mismatch. It seems like a good idea to clear out the no-longer-needed fields, even though in principle nothing should look at them; the node ought to be indistinguishable from how it would look if we'd built a new node instead of scribbling on the old one. BuildOnConflictExcludedTargetlist randomly set the resname of some TargetEntries to "" not NULL. outfuncs/readfuncs don't distinguish those cases, and so the string will read back in as NULL ... but equalfuncs.c does distinguish. Perhaps we ought to try to make things more consistent in this area --- but it's just useless extra code space for BuildOnConflictExcludedTargetlist to not use NULL here, so I fixed it for now by making it do that. catversion bumped because the change in handling of Query.withCheckOptions affects stored rules. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 21:08:28 +02:00
* Now, plug the view query in as a subselect, converting the relation's
* original RTE to a subquery RTE.
*/
rte = rt_fetch(rt_index, parsetree->rtable);
rte->rtekind = RTE_SUBQUERY;
rte->subquery = rule_action;
Fix some minor issues exposed by outfuncs/readfuncs testing. A test patch to pass parse and plan trees through outfuncs + readfuncs exposed several issues that need to be fixed to get clean matches: Query.withCheckOptions failed to get copied; it's intentionally ignored by outfuncs/readfuncs on the grounds that it'd always be NIL anyway in stored rules. This seems less than future-proof, and it's not even saving very much, so just undo the decision and treat the field like all others. Several places that convert a view RTE into a subquery RTE, or similar manipulations, failed to clear out fields that were specific to the original RTE type and should be zero in a subquery RTE. Since readfuncs.c will leave such fields as zero, equalfuncs.c thinks the nodes are different leading to a reported mismatch. It seems like a good idea to clear out the no-longer-needed fields, even though in principle nothing should look at them; the node ought to be indistinguishable from how it would look if we'd built a new node instead of scribbling on the old one. BuildOnConflictExcludedTargetlist randomly set the resname of some TargetEntries to "" not NULL. outfuncs/readfuncs don't distinguish those cases, and so the string will read back in as NULL ... but equalfuncs.c does distinguish. Perhaps we ought to try to make things more consistent in this area --- but it's just useless extra code space for BuildOnConflictExcludedTargetlist to not use NULL here, so I fixed it for now by making it do that. catversion bumped because the change in handling of Query.withCheckOptions affects stored rules. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 21:08:28 +02:00
rte->security_barrier = RelationIsSecurityView(relation);
/* Clear fields that should not be set in a subquery RTE */
rte->relid = InvalidOid;
rte->relkind = 0;
rte->rellockmode = 0;
Fix some minor issues exposed by outfuncs/readfuncs testing. A test patch to pass parse and plan trees through outfuncs + readfuncs exposed several issues that need to be fixed to get clean matches: Query.withCheckOptions failed to get copied; it's intentionally ignored by outfuncs/readfuncs on the grounds that it'd always be NIL anyway in stored rules. This seems less than future-proof, and it's not even saving very much, so just undo the decision and treat the field like all others. Several places that convert a view RTE into a subquery RTE, or similar manipulations, failed to clear out fields that were specific to the original RTE type and should be zero in a subquery RTE. Since readfuncs.c will leave such fields as zero, equalfuncs.c thinks the nodes are different leading to a reported mismatch. It seems like a good idea to clear out the no-longer-needed fields, even though in principle nothing should look at them; the node ought to be indistinguishable from how it would look if we'd built a new node instead of scribbling on the old one. BuildOnConflictExcludedTargetlist randomly set the resname of some TargetEntries to "" not NULL. outfuncs/readfuncs don't distinguish those cases, and so the string will read back in as NULL ... but equalfuncs.c does distinguish. Perhaps we ought to try to make things more consistent in this area --- but it's just useless extra code space for BuildOnConflictExcludedTargetlist to not use NULL here, so I fixed it for now by making it do that. catversion bumped because the change in handling of Query.withCheckOptions affects stored rules. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 21:08:28 +02:00
rte->tablesample = NULL;
rte->inh = false; /* must not be set for a subquery */
/*
* We move the view's permission check data down to its rangetable. The
* checks will actually be done against the OLD entry therein.
*/
subrte = rt_fetch(PRS2_OLD_VARNO, rule_action->rtable);
Assert(subrte->relid == relation->rd_id);
subrte->requiredPerms = rte->requiredPerms;
subrte->checkAsUser = rte->checkAsUser;
subrte->selectedCols = rte->selectedCols;
subrte->insertedCols = rte->insertedCols;
subrte->updatedCols = rte->updatedCols;
subrte->extraUpdatedCols = rte->extraUpdatedCols;
rte->requiredPerms = 0; /* no permission check on subquery itself */
rte->checkAsUser = InvalidOid;
rte->selectedCols = NULL;
rte->insertedCols = NULL;
rte->updatedCols = NULL;
rte->extraUpdatedCols = NULL;
/*
* Since we allow CREATE OR REPLACE VIEW to add columns to a view, the
* rule_action might emit more columns than we expected when the current
* query was parsed. Various places expect rte->eref->colnames to be
* consistent with the non-junk output columns of the subquery, so patch
* things up if necessary by adding some dummy column names.
*/
numCols = ExecCleanTargetListLength(rule_action->targetList);
while (list_length(rte->eref->colnames) < numCols)
{
rte->eref->colnames = lappend(rte->eref->colnames,
makeString(pstrdup("?column?")));
}
return parsetree;
}
/*
Improve concurrency of foreign key locking This patch introduces two additional lock modes for tuples: "SELECT FOR KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each other, in contrast with already existing "SELECT FOR SHARE" and "SELECT FOR UPDATE". UPDATE commands that do not modify the values stored in the columns that are part of the key of the tuple now grab a SELECT FOR NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently with tuple locks of the FOR KEY SHARE variety. Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this means the concurrency improvement applies to them, which is the whole point of this patch. The added tuple lock semantics require some rejiggering of the multixact module, so that the locking level that each transaction is holding can be stored alongside its Xid. Also, multixacts now need to persist across server restarts and crashes, because they can now represent not only tuple locks, but also tuple updates. This means we need more careful tracking of lifetime of pg_multixact SLRU files; since they now persist longer, we require more infrastructure to figure out when they can be removed. pg_upgrade also needs to be careful to copy pg_multixact files over from the old server to the new, or at least part of multixact.c state, depending on the versions of the old and new servers. Tuple time qualification rules (HeapTupleSatisfies routines) need to be careful not to consider tuples with the "is multi" infomask bit set as being only locked; they might need to look up MultiXact values (i.e. possibly do pg_multixact I/O) to find out the Xid that updated a tuple, whereas they previously were assured to only use information readily available from the tuple header. This is considered acceptable, because the extra I/O would involve cases that would previously cause some commands to block waiting for concurrent transactions to finish. Another important change is the fact that locking tuples that have previously been updated causes the future versions to be marked as locked, too; this is essential for correctness of foreign key checks. This causes additional WAL-logging, also (there was previously a single WAL record for a locked tuple; now there are as many as updated copies of the tuple there exist.) With all this in place, contention related to tuples being checked by foreign key rules should be much reduced. As a bonus, the old behavior that a subtransaction grabbing a stronger tuple lock than the parent (sub)transaction held on a given tuple and later aborting caused the weaker lock to be lost, has been fixed. Many new spec files were added for isolation tester framework, to ensure overall behavior is sane. There's probably room for several more tests. There were several reviewers of this patch; in particular, Noah Misch and Andres Freund spent considerable time in it. Original idea for the patch came from Simon Riggs, after a problem report by Joel Jacobson. Most code is from me, with contributions from Marti Raudsepp, Alexander Shulgin, Noah Misch and Andres Freund. This patch was discussed in several pgsql-hackers threads; the most important start at the following message-ids: AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com 1290721684-sup-3951@alvh.no-ip.org 1294953201-sup-2099@alvh.no-ip.org 1320343602-sup-2290@alvh.no-ip.org 1339690386-sup-8927@alvh.no-ip.org 4FE5FF020200002500048A3D@gw.wicourts.gov 4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
* Recursively mark all relations used by a view as FOR [KEY] UPDATE/SHARE.
*
* This may generate an invalid query, eg if some sub-query uses an
* aggregate. We leave it to the planner to detect that.
*
* NB: this must agree with the parser's transformLockingClause() routine.
* However, unlike the parser we have to be careful not to mark a view's
* OLD and NEW rels for updating. The best way to handle that seems to be
* to scan the jointree to determine which rels are used.
*/
static void
markQueryForLocking(Query *qry, Node *jtnode,
LockClauseStrength strength, LockWaitPolicy waitPolicy,
bool pushedDown)
{
if (jtnode == NULL)
return;
if (IsA(jtnode, RangeTblRef))
{
int rti = ((RangeTblRef *) jtnode)->rtindex;
RangeTblEntry *rte = rt_fetch(rti, qry->rtable);
if (rte->rtekind == RTE_RELATION)
{
applyLockingClause(qry, rti, strength, waitPolicy, pushedDown);
rte->requiredPerms |= ACL_SELECT_FOR_UPDATE;
}
else if (rte->rtekind == RTE_SUBQUERY)
{
applyLockingClause(qry, rti, strength, waitPolicy, pushedDown);
/* FOR UPDATE/SHARE of subquery is propagated to subquery's rels */
markQueryForLocking(rte->subquery, (Node *) rte->subquery->jointree,
strength, waitPolicy, true);
}
/* other RTE types are unaffected by FOR UPDATE */
}
else if (IsA(jtnode, FromExpr))
{
FromExpr *f = (FromExpr *) jtnode;
ListCell *l;
foreach(l, f->fromlist)
markQueryForLocking(qry, lfirst(l), strength, waitPolicy, pushedDown);
}
else if (IsA(jtnode, JoinExpr))
{
JoinExpr *j = (JoinExpr *) jtnode;
markQueryForLocking(qry, j->larg, strength, waitPolicy, pushedDown);
markQueryForLocking(qry, j->rarg, strength, waitPolicy, pushedDown);
}
else
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(jtnode));
}
/*
* fireRIRonSubLink -
* Apply fireRIRrules() to each SubLink (subselect in expression) found
* in the given tree.
*
* NOTE: although this has the form of a walker, we cheat and modify the
* SubLink nodes in-place. It is caller's responsibility to ensure that
* no unwanted side-effects occur!
*
* This is unlike most of the other routines that recurse into subselects,
* because we must take control at the SubLink node in order to replace
* the SubLink's subselect link with the possibly-rewritten subquery.
*/
static bool
fireRIRonSubLink(Node *node, List *activeRIRs)
{
if (node == NULL)
return false;
if (IsA(node, SubLink))
{
SubLink *sub = (SubLink *) node;
/* Do what we came for */
sub->subselect = (Node *) fireRIRrules((Query *) sub->subselect,
activeRIRs);
/* Fall through to process lefthand args of SubLink */
}
2001-03-22 05:01:46 +01:00
/*
* Do NOT recurse into Query nodes, because fireRIRrules already processed
* subselects of subselects for us.
*/
return expression_tree_walker(node, fireRIRonSubLink,
(void *) activeRIRs);
}
/*
* fireRIRrules -
* Apply all RIR rules on each rangetable entry in the given query
*
* activeRIRs is a list of the OIDs of views we're already processing RIR
* rules for, used to detect/reject recursion.
*/
static Query *
fireRIRrules(Query *parsetree, List *activeRIRs)
{
int origResultRelation = parsetree->resultRelation;
int rt_index;
ListCell *lc;
/*
* Expand SEARCH and CYCLE clauses in CTEs.
*
* This is just a convenient place to do this, since we are already
* looking at each Query.
*/
foreach(lc, parsetree->cteList)
{
CommonTableExpr *cte = lfirst_node(CommonTableExpr, lc);
if (cte->search_clause || cte->cycle_clause)
{
cte = rewriteSearchAndCycle(cte);
lfirst(lc) = cte;
}
}
/*
* don't try to convert this into a foreach loop, because rtable list can
* get changed each time through...
*/
rt_index = 0;
while (rt_index < list_length(parsetree->rtable))
{
RangeTblEntry *rte;
Relation rel;
List *locks;
RuleLock *rules;
RewriteRule *rule;
int i;
++rt_index;
rte = rt_fetch(rt_index, parsetree->rtable);
/*
* A subquery RTE can't have associated rules, so there's nothing to
* do to this level of the query, but we must recurse into the
* subquery to expand any rule references in it.
*/
if (rte->rtekind == RTE_SUBQUERY)
{
rte->subquery = fireRIRrules(rte->subquery, activeRIRs);
continue;
}
/*
* Joins and other non-relation RTEs can be ignored completely.
*/
if (rte->rtekind != RTE_RELATION)
continue;
/*
* Always ignore RIR rules for materialized views referenced in
* queries. (This does not prevent refreshing MVs, since they aren't
* referenced in their own query definitions.)
*
* Note: in the future we might want to allow MVs to be conditionally
* expanded as if they were regular views, if they are not scannable.
* In that case this test would need to be postponed till after we've
* opened the rel, so that we could check its state.
*/
if (rte->relkind == RELKIND_MATVIEW)
continue;
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *. When expanding an updatable view that is an INSERT's target, the rewriter failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally worked if the view was just "SELECT * FROM ...", as the transformation would be a no-op in that case. With more complicated view targetlists, this omission would often lead to "attribute ... has the wrong type" errors or even crashes, as reported by Mario De Frutos Dieguez. Fix by adding code to rewriteTargetView to fix up the data structure correctly. The easiest way to update the exclRelTlist list is to rebuild it from scratch looking at the new target relation, so factor the code for that out of transformOnConflictClause to make it sharable. In passing, avoid duplicate permissions checks against the EXCLUDED pseudo-relation, and prevent useless view expansion of that relation's dummy RTE. The latter is only known to happen (after this patch) in cases where the query would fail later due to not having any INSTEAD OF triggers for the view. But by exactly that token, it would create an unintended and very poorly tested state of the query data structure, so it seems like a good idea to prevent it from happening at all. This has been broken since ON CONFLICT was introduced, so back-patch to 9.5. Dean Rasheed, based on an earlier patch by Amit Langote; comment-kibitzing and back-patching by me Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
/*
* In INSERT ... ON CONFLICT, ignore the EXCLUDED pseudo-relation;
* even if it points to a view, we needn't expand it, and should not
* because we want the RTE to remain of RTE_RELATION type. Otherwise,
* it would get changed to RTE_SUBQUERY type, which is an
* untested/unsupported situation.
*/
if (parsetree->onConflict &&
rt_index == parsetree->onConflict->exclRelIndex)
continue;
/*
* If the table is not referenced in the query, then we ignore it.
* This prevents infinite expansion loop due to new rtable entries
* inserted by expansion of a rule. A table is referenced if it is
* part of the join set (a source table), or is referenced by any Var
* nodes, or is the result table.
*/
if (rt_index != parsetree->resultRelation &&
!rangeTableEntry_used((Node *) parsetree, rt_index, 0))
continue;
1999-05-25 18:15:34 +02:00
/*
* Also, if this is a new result relation introduced by
* ApplyRetrieveRule, we don't want to do anything more with it.
*/
if (rt_index == parsetree->resultRelation &&
rt_index != origResultRelation)
continue;
/*
* We can use NoLock here since either the parser or
* AcquireRewriteLocks should have locked the rel already.
*/
rel = table_open(rte->relid, NoLock);
/*
* Collect the RIR rules that we must apply
*/
rules = rel->rd_rules;
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
if (rules != NULL)
{
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
locks = NIL;
for (i = 0; i < rules->numLocks; i++)
{
rule = rules->rules[i];
if (rule->event != CMD_SELECT)
continue;
1999-05-25 18:15:34 +02:00
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
locks = lappend(locks, rule);
}
/*
* If we found any, apply them --- but first check for recursion!
*/
if (locks != NIL)
{
ListCell *l;
if (list_member_oid(activeRIRs, RelationGetRelid(rel)))
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("infinite recursion detected in rules for relation \"%s\"",
RelationGetRelationName(rel))));
Avoid using lcons and list_delete_first where it's easy to do so. Formerly, lcons was about the same speed as lappend, but with the new List implementation, that's not so; with a long List, data movement imposes an O(N) cost on lcons and list_delete_first, but not lappend. Hence, invent list_delete_last with semantics parallel to list_delete_first (but O(1) cost), and change various places to use lappend and list_delete_last where this can be done without much violence to the code logic. There are quite a few places that construct result lists using lcons not lappend. Some have semantic rationales for that; I added comments about it to a couple that didn't have them already. In many such places though, I think the coding is that way only because back in the dark ages lcons was faster than lappend. Hence, switch to lappend where this can be done without causing semantic changes. In ExecInitExprRec(), this results in aggregates and window functions that are in the same plan node being executed in a different order than before. Generally, the executions of such functions ought to be independent of each other, so this shouldn't result in visibly different query results. But if you push it, as one regression test case does, you can show that the order is different. The new order seems saner; it's closer to the order of the functions in the query text. And we never documented or promised anything about this, anyway. Also, in gistfinishsplit(), don't bother building a reverse-order list; it's easy now to iterate backwards through the original list. It'd be possible to go further towards removing uses of lcons and list_delete_first, but it'd require more extensive logic changes, and I'm not convinced it's worth it. Most of the remaining uses deal with queues that probably never get long enough to be worth sweating over. (Actually, I doubt that any of the changes in this patch will have measurable performance effects either. But better to have good examples than bad ones in the code base.) Patch by me, thanks to David Rowley and Daniel Gustafsson for review. Discussion: https://postgr.es/m/21272.1563318411@sss.pgh.pa.us
2019-07-17 17:15:28 +02:00
activeRIRs = lappend_oid(activeRIRs, RelationGetRelid(rel));
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
foreach(l, locks)
{
rule = lfirst(l);
parsetree = ApplyRetrieveRule(parsetree,
rule,
rt_index,
rel,
activeRIRs);
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
}
Avoid using lcons and list_delete_first where it's easy to do so. Formerly, lcons was about the same speed as lappend, but with the new List implementation, that's not so; with a long List, data movement imposes an O(N) cost on lcons and list_delete_first, but not lappend. Hence, invent list_delete_last with semantics parallel to list_delete_first (but O(1) cost), and change various places to use lappend and list_delete_last where this can be done without much violence to the code logic. There are quite a few places that construct result lists using lcons not lappend. Some have semantic rationales for that; I added comments about it to a couple that didn't have them already. In many such places though, I think the coding is that way only because back in the dark ages lcons was faster than lappend. Hence, switch to lappend where this can be done without causing semantic changes. In ExecInitExprRec(), this results in aggregates and window functions that are in the same plan node being executed in a different order than before. Generally, the executions of such functions ought to be independent of each other, so this shouldn't result in visibly different query results. But if you push it, as one regression test case does, you can show that the order is different. The new order seems saner; it's closer to the order of the functions in the query text. And we never documented or promised anything about this, anyway. Also, in gistfinishsplit(), don't bother building a reverse-order list; it's easy now to iterate backwards through the original list. It'd be possible to go further towards removing uses of lcons and list_delete_first, but it'd require more extensive logic changes, and I'm not convinced it's worth it. Most of the remaining uses deal with queues that probably never get long enough to be worth sweating over. (Actually, I doubt that any of the changes in this patch will have measurable performance effects either. But better to have good examples than bad ones in the code base.) Patch by me, thanks to David Rowley and Daniel Gustafsson for review. Discussion: https://postgr.es/m/21272.1563318411@sss.pgh.pa.us
2019-07-17 17:15:28 +02:00
activeRIRs = list_delete_last(activeRIRs);
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
}
}
table_close(rel, NoLock);
}
/* Recurse into subqueries in WITH */
foreach(lc, parsetree->cteList)
{
CommonTableExpr *cte = (CommonTableExpr *) lfirst(lc);
cte->ctequery = (Node *)
fireRIRrules((Query *) cte->ctequery, activeRIRs);
}
/*
* Recurse into sublink subqueries, too. But we already did the ones in
* the rtable and cteList.
*/
if (parsetree->hasSubLinks)
query_tree_walker(parsetree, fireRIRonSubLink, (void *) activeRIRs,
QTW_IGNORE_RC_SUBQUERIES);
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
/*
2021-04-21 08:14:43 +02:00
* Apply any row-level security policies. We do this last because it
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
* requires special recursion detection if the new quals have sublink
* subqueries, and if we did it in the loop above query_tree_walker would
* then recurse into those quals a second time.
*/
rt_index = 0;
foreach(lc, parsetree->rtable)
{
RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc);
Relation rel;
List *securityQuals;
List *withCheckOptions;
bool hasRowSecurity;
bool hasSubLinks;
++rt_index;
/* Only normal relations can have RLS policies */
if (rte->rtekind != RTE_RELATION ||
(rte->relkind != RELKIND_RELATION &&
rte->relkind != RELKIND_PARTITIONED_TABLE))
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
continue;
rel = table_open(rte->relid, NoLock);
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
/*
* Fetch any new security quals that must be applied to this RTE.
*/
get_row_security_policies(parsetree, rte, rt_index,
&securityQuals, &withCheckOptions,
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
&hasRowSecurity, &hasSubLinks);
if (securityQuals != NIL || withCheckOptions != NIL)
{
if (hasSubLinks)
{
acquireLocksOnSubLinks_context context;
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
/*
* Recursively process the new quals, checking for infinite
* recursion.
*/
if (list_member_oid(activeRIRs, RelationGetRelid(rel)))
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("infinite recursion detected in policy for relation \"%s\"",
RelationGetRelationName(rel))));
Avoid using lcons and list_delete_first where it's easy to do so. Formerly, lcons was about the same speed as lappend, but with the new List implementation, that's not so; with a long List, data movement imposes an O(N) cost on lcons and list_delete_first, but not lappend. Hence, invent list_delete_last with semantics parallel to list_delete_first (but O(1) cost), and change various places to use lappend and list_delete_last where this can be done without much violence to the code logic. There are quite a few places that construct result lists using lcons not lappend. Some have semantic rationales for that; I added comments about it to a couple that didn't have them already. In many such places though, I think the coding is that way only because back in the dark ages lcons was faster than lappend. Hence, switch to lappend where this can be done without causing semantic changes. In ExecInitExprRec(), this results in aggregates and window functions that are in the same plan node being executed in a different order than before. Generally, the executions of such functions ought to be independent of each other, so this shouldn't result in visibly different query results. But if you push it, as one regression test case does, you can show that the order is different. The new order seems saner; it's closer to the order of the functions in the query text. And we never documented or promised anything about this, anyway. Also, in gistfinishsplit(), don't bother building a reverse-order list; it's easy now to iterate backwards through the original list. It'd be possible to go further towards removing uses of lcons and list_delete_first, but it'd require more extensive logic changes, and I'm not convinced it's worth it. Most of the remaining uses deal with queues that probably never get long enough to be worth sweating over. (Actually, I doubt that any of the changes in this patch will have measurable performance effects either. But better to have good examples than bad ones in the code base.) Patch by me, thanks to David Rowley and Daniel Gustafsson for review. Discussion: https://postgr.es/m/21272.1563318411@sss.pgh.pa.us
2019-07-17 17:15:28 +02:00
activeRIRs = lappend_oid(activeRIRs, RelationGetRelid(rel));
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
/*
* get_row_security_policies just passed back securityQuals
* and/or withCheckOptions, and there were SubLinks, make sure
* we lock any relations which are referenced.
*
* These locks would normally be acquired by the parser, but
* securityQuals and withCheckOptions are added post-parsing.
*/
context.for_execute = true;
(void) acquireLocksOnSubLinks((Node *) securityQuals, &context);
(void) acquireLocksOnSubLinks((Node *) withCheckOptions,
&context);
/*
* Now that we have the locks on anything added by
* get_row_security_policies, fire any RIR rules for them.
*/
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
expression_tree_walker((Node *) securityQuals,
fireRIRonSubLink, (void *) activeRIRs);
expression_tree_walker((Node *) withCheckOptions,
fireRIRonSubLink, (void *) activeRIRs);
Avoid using lcons and list_delete_first where it's easy to do so. Formerly, lcons was about the same speed as lappend, but with the new List implementation, that's not so; with a long List, data movement imposes an O(N) cost on lcons and list_delete_first, but not lappend. Hence, invent list_delete_last with semantics parallel to list_delete_first (but O(1) cost), and change various places to use lappend and list_delete_last where this can be done without much violence to the code logic. There are quite a few places that construct result lists using lcons not lappend. Some have semantic rationales for that; I added comments about it to a couple that didn't have them already. In many such places though, I think the coding is that way only because back in the dark ages lcons was faster than lappend. Hence, switch to lappend where this can be done without causing semantic changes. In ExecInitExprRec(), this results in aggregates and window functions that are in the same plan node being executed in a different order than before. Generally, the executions of such functions ought to be independent of each other, so this shouldn't result in visibly different query results. But if you push it, as one regression test case does, you can show that the order is different. The new order seems saner; it's closer to the order of the functions in the query text. And we never documented or promised anything about this, anyway. Also, in gistfinishsplit(), don't bother building a reverse-order list; it's easy now to iterate backwards through the original list. It'd be possible to go further towards removing uses of lcons and list_delete_first, but it'd require more extensive logic changes, and I'm not convinced it's worth it. Most of the remaining uses deal with queues that probably never get long enough to be worth sweating over. (Actually, I doubt that any of the changes in this patch will have measurable performance effects either. But better to have good examples than bad ones in the code base.) Patch by me, thanks to David Rowley and Daniel Gustafsson for review. Discussion: https://postgr.es/m/21272.1563318411@sss.pgh.pa.us
2019-07-17 17:15:28 +02:00
activeRIRs = list_delete_last(activeRIRs);
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
}
/*
* Add the new security barrier quals to the start of the RTE's
* list so that they get applied before any existing barrier quals
* (which would have come from a security-barrier view, and should
* get lower priority than RLS conditions on the table itself).
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
*/
rte->securityQuals = list_concat(securityQuals,
rte->securityQuals);
parsetree->withCheckOptions = list_concat(withCheckOptions,
parsetree->withCheckOptions);
}
/*
2021-04-21 08:14:43 +02:00
* Make sure the query is marked correctly if row-level security
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
* applies, or if the new quals had sublinks.
*/
if (hasRowSecurity)
parsetree->hasRowSecurity = true;
if (hasSubLinks)
parsetree->hasSubLinks = true;
table_close(rel, NoLock);
RLS fixes, new hooks, and new test module In prepend_row_security_policies(), defaultDeny was always true, so if there were any hook policies, the RLS policies on the table would just get discarded. Fixed to start off with defaultDeny as false and then properly set later if we detect that only the default deny policy exists for the internal policies. The infinite recursion detection in fireRIRrules() didn't properly manage the activeRIRs list in the case of WCOs, so it would incorrectly report infinite recusion if the same relation with RLS appeared more than once in the rtable, for example "UPDATE t ... FROM t ...". Further, the RLS expansion code in fireRIRrules() was handling RLS in the main loop through the rtable, which lead to RTEs being visited twice if they contained sublink subqueries, which prepend_row_security_policies() attempted to handle by exiting early if the RTE already had securityQuals. That doesn't work, however, since if the query involved a security barrier view on top of a table with RLS, the RTE would already have securityQuals (from the view) by the time fireRIRrules() was invoked, and so the table's RLS policies would be ignored. This is fixed in fireRIRrules() by handling RLS in a separate loop at the end, after dealing with any other sublink subqueries, thus ensuring that each RTE is only visited once for RLS expansion. The inheritance planner code didn't correctly handle non-target relations with RLS, which would get turned into subqueries during planning. Thus an update of the form "UPDATE t1 ... FROM t2 ..." where t1 has inheritance and t2 has RLS quals would fail. Fix by making sure to copy in and update the securityQuals when they exist for non-target relations. process_policies() was adding WCOs to non-target relations, which is unnecessary, and could lead to a lot of wasted time in the rewriter and the planner. Fix by only adding WCO policies when working on the result relation. Also in process_policies, we should be copying the USING policies to the WITH CHECK policies on a per-policy basis, fix by moving the copying up into the per-policy loop. Lastly, as noted by Dean, we were simply adding policies returned by the hook provided to the list of quals being AND'd, meaning that they would actually restrict records returned and there was no option to have internal policies and hook-based policies work together permissively (as all internal policies currently work). Instead, explicitly add support for both permissive and restrictive policies by having a hook for each and combining the results appropriately. To ensure this is all done correctly, add a new test module (test_rls_hooks) to test the various combinations of internal, permissive, and restrictive hook policies. Largely from Dean Rasheed (thanks!): CAEZATCVmFUfUOwwhnBTcgi6AquyjQ0-1fyKd0T3xBWJvn+xsFA@mail.gmail.com Author: Dean Rasheed, though I added the new hooks and test module.
2015-04-22 18:01:06 +02:00
}
return parsetree;
}
/*
* Modify the given query by adding 'AND rule_qual IS NOT TRUE' to its
* qualification. This is used to generate suitable "else clauses" for
* conditional INSTEAD rules. (Unfortunately we must use "x IS NOT TRUE",
* not just "NOT x" which the planner is much smarter about, else we will
* do the wrong thing when the qual evaluates to NULL.)
*
* The rule_qual may contain references to OLD or NEW. OLD references are
* replaced by references to the specified rt_index (the relation that the
* rule applies to). NEW references are only possible for INSERT and UPDATE
* queries on the relation itself, and so they should be replaced by copies
* of the related entries in the query's own targetlist.
*/
static Query *
CopyAndAddInvertedQual(Query *parsetree,
Node *rule_qual,
int rt_index,
CmdType event)
{
/* Don't scribble on the passed qual (it's in the relcache!) */
Node *new_qual = copyObject(rule_qual);
acquireLocksOnSubLinks_context context;
context.for_execute = true;
/*
* In case there are subqueries in the qual, acquire necessary locks and
* fix any deleted JOIN RTE entries. (This is somewhat redundant with
* rewriteRuleAction, but not entirely ... consider restructuring so that
* we only need to process the qual this way once.)
*/
(void) acquireLocksOnSubLinks(new_qual, &context);
/* Fix references to OLD */
ChangeVarNodes(new_qual, PRS2_OLD_VARNO, rt_index, 0);
/* Fix references to NEW */
if (event == CMD_INSERT || event == CMD_UPDATE)
new_qual = ReplaceVarsFromTargetList(new_qual,
PRS2_NEW_VARNO,
0,
rt_fetch(rt_index,
parsetree->rtable),
parsetree->targetList,
(event == CMD_UPDATE) ?
REPLACEVARS_CHANGE_VARNO :
REPLACEVARS_SUBSTITUTE_NULL,
rt_index,
&parsetree->hasSubLinks);
/* And attach the fixed qual */
AddInvertedQual(parsetree, new_qual);
return parsetree;
}
/*
* fireRules -
* Iterate through rule locks applying rules.
*
* Input arguments:
* parsetree - original query
* rt_index - RT index of result relation in original query
* event - type of rule event
* locks - list of rules to fire
* Output arguments:
* *instead_flag - set true if any unqualified INSTEAD rule is found
* (must be initialized to false)
* *returning_flag - set true if we rewrite RETURNING clause in any rule
* (must be initialized to false)
* *qual_product - filled with modified original query if any qualified
* INSTEAD rule is found (must be initialized to NULL)
* Return value:
* list of rule actions adjusted for use with this query
*
* Qualified INSTEAD rules generate their action with the qualification
* condition added. They also generate a modified version of the original
* query with the negated qualification added, so that it will run only for
* rows that the qualified action doesn't act on. (If there are multiple
* qualified INSTEAD rules, we AND all the negated quals onto a single
* modified original query.) We won't execute the original, unmodified
* query if we find either qualified or unqualified INSTEAD rules. If
* we find both, the modified original query is discarded too.
*/
static List *
fireRules(Query *parsetree,
int rt_index,
CmdType event,
List *locks,
bool *instead_flag,
bool *returning_flag,
Query **qual_product)
{
List *results = NIL;
ListCell *l;
foreach(l, locks)
{
RewriteRule *rule_lock = (RewriteRule *) lfirst(l);
Node *event_qual = rule_lock->qual;
List *actions = rule_lock->actions;
QuerySource qsrc;
ListCell *r;
/* Determine correct QuerySource value for actions */
if (rule_lock->isInstead)
{
if (event_qual != NULL)
qsrc = QSRC_QUAL_INSTEAD_RULE;
else
{
qsrc = QSRC_INSTEAD_RULE;
*instead_flag = true; /* report unqualified INSTEAD */
}
}
else
qsrc = QSRC_NON_INSTEAD_RULE;
if (qsrc == QSRC_QUAL_INSTEAD_RULE)
{
/*
* If there are INSTEAD rules with qualifications, the original
* query is still performed. But all the negated rule
* qualifications of the INSTEAD rules are added so it does its
* actions only in cases where the rule quals of all INSTEAD rules
* are false. Think of it as the default action in a case. We save
* this in *qual_product so RewriteQuery() can add it to the query
* list after we mangled it up enough.
*
* If we have already found an unqualified INSTEAD rule, then
* *qual_product won't be used, so don't bother building it.
*/
if (!*instead_flag)
{
if (*qual_product == NULL)
*qual_product = copyObject(parsetree);
*qual_product = CopyAndAddInvertedQual(*qual_product,
event_qual,
rt_index,
event);
}
}
/* Now process the rule's actions and add them to the result list */
foreach(r, actions)
{
Query *rule_action = lfirst(r);
if (rule_action->commandType == CMD_NOTHING)
continue;
rule_action = rewriteRuleAction(parsetree, rule_action,
event_qual, rt_index, event,
returning_flag);
rule_action->querySource = qsrc;
rule_action->canSetTag = false; /* might change later */
results = lappend(results, rule_action);
}
}
return results;
}
/*
* get_view_query - get the Query from a view's _RETURN rule.
*
* Caller should have verified that the relation is a view, and therefore
* we should find an ON SELECT action.
*
* Note that the pointer returned is into the relcache and therefore must
* be treated as read-only to the caller and not modified or scribbled on.
*/
Query *
get_view_query(Relation view)
{
int i;
Assert(view->rd_rel->relkind == RELKIND_VIEW);
for (i = 0; i < view->rd_rules->numLocks; i++)
{
RewriteRule *rule = view->rd_rules->rules[i];
if (rule->event == CMD_SELECT)
{
/* A _RETURN rule should have only one action */
if (list_length(rule->actions) != 1)
elog(ERROR, "invalid _RETURN rule action specification");
return (Query *) linitial(rule->actions);
}
}
elog(ERROR, "failed to find _RETURN rule for view");
return NULL; /* keep compiler quiet */
}
/*
* view_has_instead_trigger - does view have an INSTEAD OF trigger for event?
*
* If it does, we don't want to treat it as auto-updatable. This test can't
* be folded into view_query_is_auto_updatable because it's not an error
* condition.
*/
static bool
view_has_instead_trigger(Relation view, CmdType event)
{
TriggerDesc *trigDesc = view->trigdesc;
switch (event)
{
case CMD_INSERT:
if (trigDesc && trigDesc->trig_insert_instead_row)
return true;
break;
case CMD_UPDATE:
if (trigDesc && trigDesc->trig_update_instead_row)
return true;
break;
case CMD_DELETE:
if (trigDesc && trigDesc->trig_delete_instead_row)
return true;
break;
default:
elog(ERROR, "unrecognized CmdType: %d", (int) event);
break;
}
return false;
}
/*
* view_col_is_auto_updatable - test whether the specified column of a view
* is auto-updatable. Returns NULL (if the column can be updated) or a message
* string giving the reason that it cannot be.
*
* The returned string has not been translated; if it is shown as an error
* message, the caller should apply _() to translate it.
*
* Note that the checks performed here are local to this view. We do not check
* whether the referenced column of the underlying base relation is updatable.
*/
static const char *
view_col_is_auto_updatable(RangeTblRef *rtr, TargetEntry *tle)
{
Var *var = (Var *) tle->expr;
/*
* For now, the only updatable columns we support are those that are Vars
* referring to user columns of the underlying base relation.
*
* The view targetlist may contain resjunk columns (e.g., a view defined
* like "SELECT * FROM t ORDER BY a+b" is auto-updatable) but such columns
* are not auto-updatable, and in fact should never appear in the outer
* query's targetlist.
*/
if (tle->resjunk)
return gettext_noop("Junk view columns are not updatable.");
if (!IsA(var, Var) ||
var->varno != rtr->rtindex ||
var->varlevelsup != 0)
return gettext_noop("View columns that are not columns of their base relation are not updatable.");
if (var->varattno < 0)
return gettext_noop("View columns that refer to system columns are not updatable.");
if (var->varattno == 0)
return gettext_noop("View columns that return whole-row references are not updatable.");
return NULL; /* the view column is updatable */
}
/*
* view_query_is_auto_updatable - test whether the specified view definition
* represents an auto-updatable view. Returns NULL (if the view can be updated)
* or a message string giving the reason that it cannot be.
* The returned string has not been translated; if it is shown as an error
* message, the caller should apply _() to translate it.
*
* If check_cols is true, the view is required to have at least one updatable
* column (necessary for INSERT/UPDATE). Otherwise the view's columns are not
* checked for updatability. See also view_cols_are_auto_updatable.
*
* Note that the checks performed here are only based on the view definition.
* We do not check whether any base relations referred to by the view are
* updatable.
*/
const char *
view_query_is_auto_updatable(Query *viewquery, bool check_cols)
{
RangeTblRef *rtr;
RangeTblEntry *base_rte;
/*----------
* Check if the view is simply updatable. According to SQL-92 this means:
* - No DISTINCT clause.
* - Each TLE is a column reference, and each column appears at most once.
* - FROM contains exactly one base relation.
* - No GROUP BY or HAVING clauses.
* - No set operations (UNION, INTERSECT or EXCEPT).
* - No sub-queries in the WHERE clause that reference the target table.
*
* We ignore that last restriction since it would be complex to enforce
* and there isn't any actual benefit to disallowing sub-queries. (The
* semantic issues that the standard is presumably concerned about don't
* arise in Postgres, since any such sub-query will not see any updates
* executed by the outer query anyway, thanks to MVCC snapshotting.)
*
* We also relax the second restriction by supporting part of SQL:1999
* feature T111, which allows for a mix of updatable and non-updatable
* columns, provided that an INSERT or UPDATE doesn't attempt to assign to
* a non-updatable column.
*
* In addition we impose these constraints, involving features that are
* not part of SQL-92:
* - No CTEs (WITH clauses).
* - No OFFSET or LIMIT clauses (this matches a SQL:2008 restriction).
* - No system columns (including whole-row references) in the tlist.
* - No window functions in the tlist.
* - No set-returning functions in the tlist.
*
* Note that we do these checks without recursively expanding the view.
* If the base relation is a view, we'll recursively deal with it later.
*----------
*/
if (viewquery->distinctClause != NIL)
return gettext_noop("Views containing DISTINCT are not automatically updatable.");
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
if (viewquery->groupClause != NIL || viewquery->groupingSets)
return gettext_noop("Views containing GROUP BY are not automatically updatable.");
if (viewquery->havingQual != NULL)
return gettext_noop("Views containing HAVING are not automatically updatable.");
if (viewquery->setOperations != NULL)
2013-04-14 17:12:30 +02:00
return gettext_noop("Views containing UNION, INTERSECT, or EXCEPT are not automatically updatable.");
if (viewquery->cteList != NIL)
return gettext_noop("Views containing WITH are not automatically updatable.");
if (viewquery->limitOffset != NULL || viewquery->limitCount != NULL)
return gettext_noop("Views containing LIMIT or OFFSET are not automatically updatable.");
/*
* We must not allow window functions or set returning functions in the
* targetlist. Otherwise we might end up inserting them into the quals of
* the main query. We must also check for aggregates in the targetlist in
* case they appear without a GROUP BY.
*
* These restrictions ensure that each row of the view corresponds to a
* unique row in the underlying base relation.
*/
if (viewquery->hasAggs)
2014-08-29 06:01:34 +02:00
return gettext_noop("Views that return aggregate functions are not automatically updatable.");
if (viewquery->hasWindowFuncs)
2014-08-29 06:01:34 +02:00
return gettext_noop("Views that return window functions are not automatically updatable.");
Improve parser's and planner's handling of set-returning functions. Teach the parser to reject misplaced set-returning functions during parse analysis using p_expr_kind, in much the same way as we do for aggregates and window functions (cf commit eaccfded9). While this isn't complete (it misses nesting-based restrictions), it's much better than the previous error reporting for such cases, and it allows elimination of assorted ad-hoc expression_returns_set() error checks. We could add nesting checks later if it seems important to catch all cases at parse time. There is one case the parser will now throw error for although previous versions allowed it, which is SRFs in the tlist of an UPDATE. That never behaved sensibly (since it's ill-defined which generated row should be used to perform the update) and it's hard to see why it should not be treated as an error. It's a release-note-worthy change though. Also, add a new Query field hasTargetSRFs reporting whether there are any SRFs in the targetlist (including GROUP BY/ORDER BY expressions). The parser can now set that basically for free during parse analysis, and we can use it in a number of places to avoid expression_returns_set searches. (There will be more such checks soon.) In some places, this allows decontorting the logic since it's no longer expensive to check for SRFs in the tlist --- so I made the checks parallel to the handling of hasAggs/hasWindowFuncs wherever it seemed appropriate. catversion bump because adding a Query field changes stored rules. Andres Freund and Tom Lane Discussion: <24639.1473782855@sss.pgh.pa.us>
2016-09-13 19:54:24 +02:00
if (viewquery->hasTargetSRFs)
return gettext_noop("Views that return set-returning functions are not automatically updatable.");
/*
* The view query should select from a single base relation, which must be
* a table or another view.
*/
if (list_length(viewquery->jointree->fromlist) != 1)
return gettext_noop("Views that do not select from a single table or view are not automatically updatable.");
rtr = (RangeTblRef *) linitial(viewquery->jointree->fromlist);
if (!IsA(rtr, RangeTblRef))
return gettext_noop("Views that do not select from a single table or view are not automatically updatable.");
base_rte = rt_fetch(rtr->rtindex, viewquery->rtable);
if (base_rte->rtekind != RTE_RELATION ||
(base_rte->relkind != RELKIND_RELATION &&
base_rte->relkind != RELKIND_FOREIGN_TABLE &&
base_rte->relkind != RELKIND_VIEW &&
base_rte->relkind != RELKIND_PARTITIONED_TABLE))
return gettext_noop("Views that do not select from a single table or view are not automatically updatable.");
if (base_rte->tablesample)
return gettext_noop("Views containing TABLESAMPLE are not automatically updatable.");
/*
* Check that the view has at least one updatable column. This is required
* for INSERT/UPDATE but not for DELETE.
*/
if (check_cols)
{
ListCell *cell;
bool found;
found = false;
foreach(cell, viewquery->targetList)
{
TargetEntry *tle = (TargetEntry *) lfirst(cell);
if (view_col_is_auto_updatable(rtr, tle) == NULL)
{
found = true;
break;
}
}
if (!found)
return gettext_noop("Views that have no updatable columns are not automatically updatable.");
}
return NULL; /* the view is updatable */
}
/*
* view_cols_are_auto_updatable - test whether all of the required columns of
* an auto-updatable view are actually updatable. Returns NULL (if all the
* required columns can be updated) or a message string giving the reason that
* they cannot be.
*
* The returned string has not been translated; if it is shown as an error
* message, the caller should apply _() to translate it.
*
* This should be used for INSERT/UPDATE to ensure that we don't attempt to
* assign to any non-updatable columns.
*
* Additionally it may be used to retrieve the set of updatable columns in the
* view, or if one or more of the required columns is not updatable, the name
* of the first offending non-updatable column.
*
* The caller must have already verified that this is an auto-updatable view
* using view_query_is_auto_updatable.
*
* Note that the checks performed here are only based on the view definition.
* We do not check whether the referenced columns of the base relation are
* updatable.
*/
static const char *
view_cols_are_auto_updatable(Query *viewquery,
Bitmapset *required_cols,
Bitmapset **updatable_cols,
char **non_updatable_col)
{
RangeTblRef *rtr;
AttrNumber col;
ListCell *cell;
/*
* The caller should have verified that this view is auto-updatable and so
* there should be a single base relation.
*/
Assert(list_length(viewquery->jointree->fromlist) == 1);
rtr = linitial_node(RangeTblRef, viewquery->jointree->fromlist);
/* Initialize the optional return values */
if (updatable_cols != NULL)
*updatable_cols = NULL;
if (non_updatable_col != NULL)
*non_updatable_col = NULL;
/* Test each view column for updatability */
col = -FirstLowInvalidHeapAttributeNumber;
foreach(cell, viewquery->targetList)
{
TargetEntry *tle = (TargetEntry *) lfirst(cell);
const char *col_update_detail;
col++;
col_update_detail = view_col_is_auto_updatable(rtr, tle);
if (col_update_detail == NULL)
{
/* The column is updatable */
if (updatable_cols != NULL)
*updatable_cols = bms_add_member(*updatable_cols, col);
}
else if (bms_is_member(col, required_cols))
{
/* The required column is not updatable */
if (non_updatable_col != NULL)
*non_updatable_col = tle->resname;
return col_update_detail;
}
}
return NULL; /* all the required view columns are updatable */
}
/*
* relation_is_updatable - determine which update events the specified
* relation supports.
*
* Note that views may contain a mix of updatable and non-updatable columns.
* For a view to support INSERT/UPDATE it must have at least one updatable
* column, but there is no such restriction for DELETE. If include_cols is
* non-NULL, then only the specified columns are considered when testing for
* updatability.
*
* Unlike the preceding functions, this does recurse to look at a view's
* base relations, so it needs to detect recursion. To do that, we pass
* a list of currently-considered outer relations. External callers need
* only pass NIL.
*
* This is used for the information_schema views, which have separate concepts
* of "updatable" and "trigger updatable". A relation is "updatable" if it
* can be updated without the need for triggers (either because it has a
* suitable RULE, or because it is simple enough to be automatically updated).
* A relation is "trigger updatable" if it has a suitable INSTEAD OF trigger.
* The SQL standard regards this as not necessarily updatable, presumably
* because there is no way of knowing what the trigger will actually do.
* The information_schema views therefore call this function with
* include_triggers = false. However, other callers might only care whether
* data-modifying SQL will work, so they can pass include_triggers = true
* to have trigger updatability included in the result.
*
* The return value is a bitmask of rule event numbers indicating which of
* the INSERT, UPDATE and DELETE operations are supported. (We do it this way
* so that we can test for UPDATE plus DELETE support in a single call.)
*/
int
relation_is_updatable(Oid reloid,
List *outer_reloids,
bool include_triggers,
Bitmapset *include_cols)
{
int events = 0;
Relation rel;
RuleLock *rulelocks;
#define ALL_EVENTS ((1 << CMD_INSERT) | (1 << CMD_UPDATE) | (1 << CMD_DELETE))
/* Since this function recurses, it could be driven to stack overflow */
check_stack_depth();
rel = try_relation_open(reloid, AccessShareLock);
/*
* If the relation doesn't exist, return zero rather than throwing an
* error. This is helpful since scanning an information_schema view under
* MVCC rules can result in referencing rels that have actually been
* deleted already.
*/
if (rel == NULL)
return 0;
/* If we detect a recursive view, report that it is not updatable */
if (list_member_oid(outer_reloids, RelationGetRelid(rel)))
{
relation_close(rel, AccessShareLock);
return 0;
}
/* If the relation is a table, it is always updatable */
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
{
relation_close(rel, AccessShareLock);
return ALL_EVENTS;
}
/* Look for unconditional DO INSTEAD rules, and note supported events */
rulelocks = rel->rd_rules;
if (rulelocks != NULL)
{
int i;
for (i = 0; i < rulelocks->numLocks; i++)
{
if (rulelocks->rules[i]->isInstead &&
rulelocks->rules[i]->qual == NULL)
{
events |= ((1 << rulelocks->rules[i]->event) & ALL_EVENTS);
}
}
/* If we have rules for all events, we're done */
if (events == ALL_EVENTS)
{
relation_close(rel, AccessShareLock);
return events;
}
}
/* Similarly look for INSTEAD OF triggers, if they are to be included */
if (include_triggers)
{
TriggerDesc *trigDesc = rel->trigdesc;
if (trigDesc)
{
if (trigDesc->trig_insert_instead_row)
events |= (1 << CMD_INSERT);
if (trigDesc->trig_update_instead_row)
events |= (1 << CMD_UPDATE);
if (trigDesc->trig_delete_instead_row)
events |= (1 << CMD_DELETE);
/* If we have triggers for all events, we're done */
if (events == ALL_EVENTS)
{
relation_close(rel, AccessShareLock);
return events;
}
}
}
/* If this is a foreign table, check which update events it supports */
if (rel->rd_rel->relkind == RELKIND_FOREIGN_TABLE)
{
FdwRoutine *fdwroutine = GetFdwRoutineForRelation(rel, false);
if (fdwroutine->IsForeignRelUpdatable != NULL)
events |= fdwroutine->IsForeignRelUpdatable(rel);
else
{
/* Assume presence of executor functions is sufficient */
if (fdwroutine->ExecForeignInsert != NULL)
events |= (1 << CMD_INSERT);
if (fdwroutine->ExecForeignUpdate != NULL)
events |= (1 << CMD_UPDATE);
if (fdwroutine->ExecForeignDelete != NULL)
events |= (1 << CMD_DELETE);
}
relation_close(rel, AccessShareLock);
return events;
}
/* Check if this is an automatically updatable view */
if (rel->rd_rel->relkind == RELKIND_VIEW)
{
Query *viewquery = get_view_query(rel);
if (view_query_is_auto_updatable(viewquery, false) == NULL)
{
Bitmapset *updatable_cols;
int auto_events;
RangeTblRef *rtr;
RangeTblEntry *base_rte;
Oid baseoid;
/*
* Determine which of the view's columns are updatable. If there
2014-03-17 12:47:28 +01:00
* are none within the set of columns we are looking at, then the
* view doesn't support INSERT/UPDATE, but it may still support
* DELETE.
*/
view_cols_are_auto_updatable(viewquery, NULL,
&updatable_cols, NULL);
if (include_cols != NULL)
updatable_cols = bms_int_members(updatable_cols, include_cols);
if (bms_is_empty(updatable_cols))
auto_events = (1 << CMD_DELETE); /* May support DELETE */
else
auto_events = ALL_EVENTS; /* May support all events */
/*
* The base relation must also support these update commands.
* Tables are always updatable, but for any other kind of base
* relation we must do a recursive check limited to the columns
* referenced by the locally updatable columns in this view.
*/
rtr = (RangeTblRef *) linitial(viewquery->jointree->fromlist);
base_rte = rt_fetch(rtr->rtindex, viewquery->rtable);
Assert(base_rte->rtekind == RTE_RELATION);
if (base_rte->relkind != RELKIND_RELATION &&
base_rte->relkind != RELKIND_PARTITIONED_TABLE)
{
baseoid = base_rte->relid;
outer_reloids = lappend_oid(outer_reloids,
RelationGetRelid(rel));
include_cols = adjust_view_column_set(updatable_cols,
viewquery->targetList);
auto_events &= relation_is_updatable(baseoid,
outer_reloids,
include_triggers,
include_cols);
outer_reloids = list_delete_last(outer_reloids);
}
events |= auto_events;
}
}
/* If we reach here, the relation may support some update commands */
relation_close(rel, AccessShareLock);
return events;
}
/*
* adjust_view_column_set - map a set of column numbers according to targetlist
*
* This is used with simply-updatable views to map column-permissions sets for
* the view columns onto the matching columns in the underlying base relation.
* The targetlist is expected to be a list of plain Vars of the underlying
* relation (as per the checks above in view_query_is_auto_updatable).
*/
static Bitmapset *
adjust_view_column_set(Bitmapset *cols, List *targetlist)
{
Bitmapset *result = NULL;
int col;
col = -1;
while ((col = bms_next_member(cols, col)) >= 0)
{
/* bit numbers are offset by FirstLowInvalidHeapAttributeNumber */
AttrNumber attno = col + FirstLowInvalidHeapAttributeNumber;
if (attno == InvalidAttrNumber)
{
/*
* There's a whole-row reference to the view. For permissions
* purposes, treat it as a reference to each column available from
* the view. (We should *not* convert this to a whole-row
* reference to the base relation, since the view may not touch
* all columns of the base relation.)
*/
ListCell *lc;
foreach(lc, targetlist)
{
TargetEntry *tle = lfirst_node(TargetEntry, lc);
Var *var;
if (tle->resjunk)
continue;
2017-02-21 17:33:07 +01:00
var = castNode(Var, tle->expr);
result = bms_add_member(result,
var->varattno - FirstLowInvalidHeapAttributeNumber);
}
}
else
{
/*
* Views do not have system columns, so we do not expect to see
* any other system attnos here. If we do find one, the error
* case will apply.
*/
TargetEntry *tle = get_tle_by_resno(targetlist, attno);
if (tle != NULL && !tle->resjunk && IsA(tle->expr, Var))
{
Var *var = (Var *) tle->expr;
result = bms_add_member(result,
var->varattno - FirstLowInvalidHeapAttributeNumber);
}
else
elog(ERROR, "attribute number %d not found in view targetlist",
attno);
}
}
return result;
}
/*
* rewriteTargetView -
* Attempt to rewrite a query where the target relation is a view, so that
* the view's base relation becomes the target relation.
*
* Note that the base relation here may itself be a view, which may or may not
* have INSTEAD OF triggers or rules to handle the update. That is handled by
* the recursion in RewriteQuery.
*/
static Query *
rewriteTargetView(Query *parsetree, Relation view)
{
Query *viewquery;
const char *auto_update_detail;
RangeTblRef *rtr;
int base_rt_index;
int new_rt_index;
RangeTblEntry *base_rte;
RangeTblEntry *view_rte;
RangeTblEntry *new_rte;
Relation base_rel;
List *view_targetlist;
ListCell *lc;
/*
* Get the Query from the view's ON SELECT rule. We're going to munge the
* Query to change the view's base relation into the target relation,
* along with various other changes along the way, so we need to make a
* copy of it (get_view_query() returns a pointer into the relcache, so we
* have to treat it as read-only).
*/
viewquery = copyObject(get_view_query(view));
/* The view must be updatable, else fail */
auto_update_detail =
view_query_is_auto_updatable(viewquery,
parsetree->commandType != CMD_DELETE);
if (auto_update_detail)
{
/* messages here should match execMain.c's CheckValidResultRel */
switch (parsetree->commandType)
{
case CMD_INSERT:
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("cannot insert into view \"%s\"",
RelationGetRelationName(view)),
errdetail_internal("%s", _(auto_update_detail)),
errhint("To enable inserting into the view, provide an INSTEAD OF INSERT trigger or an unconditional ON INSERT DO INSTEAD rule.")));
break;
case CMD_UPDATE:
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("cannot update view \"%s\"",
RelationGetRelationName(view)),
errdetail_internal("%s", _(auto_update_detail)),
errhint("To enable updating the view, provide an INSTEAD OF UPDATE trigger or an unconditional ON UPDATE DO INSTEAD rule.")));
break;
case CMD_DELETE:
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("cannot delete from view \"%s\"",
RelationGetRelationName(view)),
errdetail_internal("%s", _(auto_update_detail)),
errhint("To enable deleting from the view, provide an INSTEAD OF DELETE trigger or an unconditional ON DELETE DO INSTEAD rule.")));
break;
default:
elog(ERROR, "unrecognized CmdType: %d",
(int) parsetree->commandType);
break;
}
}
/*
* For INSERT/UPDATE the modified columns must all be updatable. Note that
* we get the modified columns from the query's targetlist, not from the
* result RTE's insertedCols and/or updatedCols set, since
* rewriteTargetListIU may have added additional targetlist entries for
* view defaults, and these must also be updatable.
*/
if (parsetree->commandType != CMD_DELETE)
{
Bitmapset *modified_cols = NULL;
char *non_updatable_col;
foreach(lc, parsetree->targetList)
{
TargetEntry *tle = (TargetEntry *) lfirst(lc);
if (!tle->resjunk)
modified_cols = bms_add_member(modified_cols,
tle->resno - FirstLowInvalidHeapAttributeNumber);
}
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
if (parsetree->onConflict)
{
foreach(lc, parsetree->onConflict->onConflictSet)
{
TargetEntry *tle = (TargetEntry *) lfirst(lc);
if (!tle->resjunk)
modified_cols = bms_add_member(modified_cols,
tle->resno - FirstLowInvalidHeapAttributeNumber);
}
}
auto_update_detail = view_cols_are_auto_updatable(viewquery,
modified_cols,
NULL,
&non_updatable_col);
if (auto_update_detail)
{
/*
* This is a different error, caused by an attempt to update a
* non-updatable column in an otherwise updatable view.
*/
switch (parsetree->commandType)
{
case CMD_INSERT:
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot insert into column \"%s\" of view \"%s\"",
non_updatable_col,
RelationGetRelationName(view)),
errdetail_internal("%s", _(auto_update_detail))));
break;
case CMD_UPDATE:
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot update column \"%s\" of view \"%s\"",
non_updatable_col,
RelationGetRelationName(view)),
errdetail_internal("%s", _(auto_update_detail))));
break;
default:
elog(ERROR, "unrecognized CmdType: %d",
(int) parsetree->commandType);
break;
}
}
}
/* Locate RTE describing the view in the outer query */
view_rte = rt_fetch(parsetree->resultRelation, parsetree->rtable);
/*
* If we get here, view_query_is_auto_updatable() has verified that the
* view contains a single base relation.
*/
Assert(list_length(viewquery->jointree->fromlist) == 1);
rtr = linitial_node(RangeTblRef, viewquery->jointree->fromlist);
base_rt_index = rtr->rtindex;
base_rte = rt_fetch(base_rt_index, viewquery->rtable);
Assert(base_rte->rtekind == RTE_RELATION);
/*
* Up to now, the base relation hasn't been touched at all in our query.
* We need to acquire lock on it before we try to do anything with it.
* (The subsequent recursive call of RewriteQuery will suppose that we
* already have the right lock!) Since it will become the query target
* relation, RowExclusiveLock is always the right thing.
*/
base_rel = table_open(base_rte->relid, RowExclusiveLock);
/*
* While we have the relation open, update the RTE's relkind, just in case
* it changed since this view was made (cf. AcquireRewriteLocks).
*/
base_rte->relkind = base_rel->rd_rel->relkind;
/*
* If the view query contains any sublink subqueries then we need to also
* acquire locks on any relations they refer to. We know that there won't
* be any subqueries in the range table or CTEs, so we can skip those, as
* in AcquireRewriteLocks.
*/
if (viewquery->hasSubLinks)
{
acquireLocksOnSubLinks_context context;
context.for_execute = true;
query_tree_walker(viewquery, acquireLocksOnSubLinks, &context,
QTW_IGNORE_RC_SUBQUERIES);
}
/*
* Create a new target RTE describing the base relation, and add it to the
* outer query's rangetable. (What's happening in the next few steps is
* very much like what the planner would do to "pull up" the view into the
* outer query. Perhaps someday we should refactor things enough so that
* we can share code with the planner.)
*
* Be sure to set rellockmode to the correct thing for the target table.
* Since we copied the whole viewquery above, we can just scribble on
* base_rte instead of copying it.
*/
new_rte = base_rte;
new_rte->rellockmode = RowExclusiveLock;
parsetree->rtable = lappend(parsetree->rtable, new_rte);
new_rt_index = list_length(parsetree->rtable);
/*
* INSERTs never inherit. For UPDATE/DELETE, we use the view query's
* inheritance flag for the base relation.
*/
if (parsetree->commandType == CMD_INSERT)
new_rte->inh = false;
/*
* Adjust the view's targetlist Vars to reference the new target RTE, ie
* make their varnos be new_rt_index instead of base_rt_index. There can
* be no Vars for other rels in the tlist, so this is sufficient to pull
* up the tlist expressions for use in the outer query. The tlist will
* provide the replacement expressions used by ReplaceVarsFromTargetList
* below.
*/
view_targetlist = viewquery->targetList;
ChangeVarNodes((Node *) view_targetlist,
base_rt_index,
new_rt_index,
0);
/*
* Mark the new target RTE for the permissions checks that we want to
* enforce against the view owner, as distinct from the query caller. At
* the relation level, require the same INSERT/UPDATE/DELETE permissions
* that the query caller needs against the view. We drop the ACL_SELECT
* bit that is presumably in new_rte->requiredPerms initially.
*
* Note: the original view RTE remains in the query's rangetable list.
* Although it will be unused in the query plan, we need it there so that
* the executor still performs appropriate permissions checks for the
* query caller's use of the view.
*/
new_rte->checkAsUser = view->rd_rel->relowner;
new_rte->requiredPerms = view_rte->requiredPerms;
/*
* Now for the per-column permissions bits.
*
* Initially, new_rte contains selectedCols permission check bits for all
* base-rel columns referenced by the view, but since the view is a SELECT
* query its insertedCols/updatedCols is empty. We set insertedCols and
* updatedCols to include all the columns the outer query is trying to
* modify, adjusting the column numbers as needed. But we leave
* selectedCols as-is, so the view owner must have read permission for all
* columns used in the view definition, even if some of them are not read
* by the outer query. We could try to limit selectedCols to only columns
* used in the transformed query, but that does not correspond to what
* happens in ordinary SELECT usage of a view: all referenced columns must
* have read permission, even if optimization finds that some of them can
* be discarded during query transformation. The flattening we're doing
* here is an optional optimization, too. (If you are unpersuaded and
* want to change this, note that applying adjust_view_column_set to
* view_rte->selectedCols is clearly *not* the right answer, since that
* neglects base-rel columns used in the view's WHERE quals.)
*
* This step needs the modified view targetlist, so we have to do things
* in this order.
*/
Assert(bms_is_empty(new_rte->insertedCols) &&
bms_is_empty(new_rte->updatedCols));
new_rte->insertedCols = adjust_view_column_set(view_rte->insertedCols,
view_targetlist);
new_rte->updatedCols = adjust_view_column_set(view_rte->updatedCols,
view_targetlist);
/*
* Move any security barrier quals from the view RTE onto the new target
* RTE. Any such quals should now apply to the new target RTE and will
* not reference the original view RTE in the rewritten query.
*/
new_rte->securityQuals = view_rte->securityQuals;
view_rte->securityQuals = NIL;
/*
* Now update all Vars in the outer query that reference the view to
* reference the appropriate column of the base relation instead.
*/
parsetree = (Query *)
ReplaceVarsFromTargetList((Node *) parsetree,
parsetree->resultRelation,
0,
view_rte,
view_targetlist,
REPLACEVARS_REPORT_ERROR,
0,
NULL);
/*
* Update all other RTI references in the query that point to the view
* (for example, parsetree->resultRelation itself) to point to the new
* base relation instead. Vars will not be affected since none of them
* reference parsetree->resultRelation any longer.
*/
ChangeVarNodes((Node *) parsetree,
parsetree->resultRelation,
new_rt_index,
0);
Assert(parsetree->resultRelation == new_rt_index);
/*
* For INSERT/UPDATE we must also update resnos in the targetlist to refer
* to columns of the base relation, since those indicate the target
* columns to be affected.
*
* Note that this destroys the resno ordering of the targetlist, but that
* will be fixed when we recurse through RewriteQuery, which will invoke
* rewriteTargetListIU again on the updated targetlist.
*/
if (parsetree->commandType != CMD_DELETE)
{
foreach(lc, parsetree->targetList)
{
TargetEntry *tle = (TargetEntry *) lfirst(lc);
TargetEntry *view_tle;
if (tle->resjunk)
continue;
view_tle = get_tle_by_resno(view_targetlist, tle->resno);
if (view_tle != NULL && !view_tle->resjunk && IsA(view_tle->expr, Var))
tle->resno = ((Var *) view_tle->expr)->varattno;
else
elog(ERROR, "attribute number %d not found in view targetlist",
tle->resno);
}
}
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *. When expanding an updatable view that is an INSERT's target, the rewriter failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally worked if the view was just "SELECT * FROM ...", as the transformation would be a no-op in that case. With more complicated view targetlists, this omission would often lead to "attribute ... has the wrong type" errors or even crashes, as reported by Mario De Frutos Dieguez. Fix by adding code to rewriteTargetView to fix up the data structure correctly. The easiest way to update the exclRelTlist list is to rebuild it from scratch looking at the new target relation, so factor the code for that out of transformOnConflictClause to make it sharable. In passing, avoid duplicate permissions checks against the EXCLUDED pseudo-relation, and prevent useless view expansion of that relation's dummy RTE. The latter is only known to happen (after this patch) in cases where the query would fail later due to not having any INSTEAD OF triggers for the view. But by exactly that token, it would create an unintended and very poorly tested state of the query data structure, so it seems like a good idea to prevent it from happening at all. This has been broken since ON CONFLICT was introduced, so back-patch to 9.5. Dean Rasheed, based on an earlier patch by Amit Langote; comment-kibitzing and back-patching by me Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
/*
* For INSERT .. ON CONFLICT .. DO UPDATE, we must also update assorted
* stuff in the onConflict data structure.
*/
if (parsetree->onConflict &&
parsetree->onConflict->action == ONCONFLICT_UPDATE)
{
Index old_exclRelIndex,
new_exclRelIndex;
Make parser rely more heavily on the ParseNamespaceItem data structure. When I added the ParseNamespaceItem data structure (in commit 5ebaaa494), it wasn't very tightly integrated into the parser's APIs. In the wake of adding p_rtindex to that struct (commit b541e9acc), there is a good reason to make more use of it: by passing around ParseNamespaceItem pointers instead of bare RTE pointers, we can get rid of various messy methods for passing back or deducing the rangetable index of an RTE during parsing. Hence, refactor the addRangeTableEntryXXX functions to build and return a ParseNamespaceItem struct, not just the RTE proper; and replace addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem rather than building one internally. Also, add per-column data (a ParseNamespaceColumn array) to each ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX, where we have column type data at hand so that it's nearly free to fill the data structure. Later, when we need to build Vars referencing RTEs, we can use the ParseNamespaceColumn info to avoid the rather expensive operations done in get_rte_attribute_type() or expandRTE(). get_rte_attribute_type() is indeed dead code now, so I've removed it. This makes for a useful improvement in parse analysis speed, around 20% in one moderately-complex test query. The ParseNamespaceColumn structs also include Var identity information (varno/varattno). That info isn't actually being used in this patch, except that p_varno == 0 is a handy test for a dropped column. A follow-on patch will make more use of it. Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
ParseNamespaceItem *new_exclNSItem;
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *. When expanding an updatable view that is an INSERT's target, the rewriter failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally worked if the view was just "SELECT * FROM ...", as the transformation would be a no-op in that case. With more complicated view targetlists, this omission would often lead to "attribute ... has the wrong type" errors or even crashes, as reported by Mario De Frutos Dieguez. Fix by adding code to rewriteTargetView to fix up the data structure correctly. The easiest way to update the exclRelTlist list is to rebuild it from scratch looking at the new target relation, so factor the code for that out of transformOnConflictClause to make it sharable. In passing, avoid duplicate permissions checks against the EXCLUDED pseudo-relation, and prevent useless view expansion of that relation's dummy RTE. The latter is only known to happen (after this patch) in cases where the query would fail later due to not having any INSTEAD OF triggers for the view. But by exactly that token, it would create an unintended and very poorly tested state of the query data structure, so it seems like a good idea to prevent it from happening at all. This has been broken since ON CONFLICT was introduced, so back-patch to 9.5. Dean Rasheed, based on an earlier patch by Amit Langote; comment-kibitzing and back-patching by me Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
RangeTblEntry *new_exclRte;
List *tmp_tlist;
/*
* Like the INSERT/UPDATE code above, update the resnos in the
* auxiliary UPDATE targetlist to refer to columns of the base
* relation.
*/
foreach(lc, parsetree->onConflict->onConflictSet)
{
TargetEntry *tle = (TargetEntry *) lfirst(lc);
TargetEntry *view_tle;
if (tle->resjunk)
continue;
view_tle = get_tle_by_resno(view_targetlist, tle->resno);
if (view_tle != NULL && !view_tle->resjunk && IsA(view_tle->expr, Var))
tle->resno = ((Var *) view_tle->expr)->varattno;
else
elog(ERROR, "attribute number %d not found in view targetlist",
tle->resno);
}
/*
* Also, create a new RTE for the EXCLUDED pseudo-relation, using the
* query's new base rel (which may well have a different column list
* from the view, hence we need a new column alias list). This should
* match transformOnConflictClause. In particular, note that the
* relkind is set to composite to signal that we're not dealing with
* an actual relation, and no permissions checks are wanted.
*/
old_exclRelIndex = parsetree->onConflict->exclRelIndex;
Make parser rely more heavily on the ParseNamespaceItem data structure. When I added the ParseNamespaceItem data structure (in commit 5ebaaa494), it wasn't very tightly integrated into the parser's APIs. In the wake of adding p_rtindex to that struct (commit b541e9acc), there is a good reason to make more use of it: by passing around ParseNamespaceItem pointers instead of bare RTE pointers, we can get rid of various messy methods for passing back or deducing the rangetable index of an RTE during parsing. Hence, refactor the addRangeTableEntryXXX functions to build and return a ParseNamespaceItem struct, not just the RTE proper; and replace addRTEtoQuery with addNSItemToQuery, which is passed a ParseNamespaceItem rather than building one internally. Also, add per-column data (a ParseNamespaceColumn array) to each ParseNamespaceItem. These arrays are built during addRangeTableEntryXXX, where we have column type data at hand so that it's nearly free to fill the data structure. Later, when we need to build Vars referencing RTEs, we can use the ParseNamespaceColumn info to avoid the rather expensive operations done in get_rte_attribute_type() or expandRTE(). get_rte_attribute_type() is indeed dead code now, so I've removed it. This makes for a useful improvement in parse analysis speed, around 20% in one moderately-complex test query. The ParseNamespaceColumn structs also include Var identity information (varno/varattno). That info isn't actually being used in this patch, except that p_varno == 0 is a handy test for a dropped column. A follow-on patch will make more use of it. Discussion: https://postgr.es/m/2461.1577764221@sss.pgh.pa.us
2020-01-02 17:29:01 +01:00
new_exclNSItem = addRangeTableEntryForRelation(make_parsestate(NULL),
base_rel,
RowExclusiveLock,
makeAlias("excluded", NIL),
false, false);
new_exclRte = new_exclNSItem->p_rte;
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *. When expanding an updatable view that is an INSERT's target, the rewriter failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally worked if the view was just "SELECT * FROM ...", as the transformation would be a no-op in that case. With more complicated view targetlists, this omission would often lead to "attribute ... has the wrong type" errors or even crashes, as reported by Mario De Frutos Dieguez. Fix by adding code to rewriteTargetView to fix up the data structure correctly. The easiest way to update the exclRelTlist list is to rebuild it from scratch looking at the new target relation, so factor the code for that out of transformOnConflictClause to make it sharable. In passing, avoid duplicate permissions checks against the EXCLUDED pseudo-relation, and prevent useless view expansion of that relation's dummy RTE. The latter is only known to happen (after this patch) in cases where the query would fail later due to not having any INSTEAD OF triggers for the view. But by exactly that token, it would create an unintended and very poorly tested state of the query data structure, so it seems like a good idea to prevent it from happening at all. This has been broken since ON CONFLICT was introduced, so back-patch to 9.5. Dean Rasheed, based on an earlier patch by Amit Langote; comment-kibitzing and back-patching by me Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
new_exclRte->relkind = RELKIND_COMPOSITE_TYPE;
new_exclRte->requiredPerms = 0;
/* other permissions fields in new_exclRte are already empty */
parsetree->rtable = lappend(parsetree->rtable, new_exclRte);
new_exclRelIndex = parsetree->onConflict->exclRelIndex =
list_length(parsetree->rtable);
/*
* Replace the targetlist for the EXCLUDED pseudo-relation with a new
* one, representing the columns from the new base relation.
*/
parsetree->onConflict->exclRelTlist =
BuildOnConflictExcludedTargetlist(base_rel, new_exclRelIndex);
/*
* Update all Vars in the ON CONFLICT clause that refer to the old
* EXCLUDED pseudo-relation. We want to use the column mappings
* defined in the view targetlist, but we need the outputs to refer to
* the new EXCLUDED pseudo-relation rather than the new target RTE.
* Also notice that "EXCLUDED.*" will be expanded using the view's
* rowtype, which seems correct.
*/
tmp_tlist = copyObject(view_targetlist);
ChangeVarNodes((Node *) tmp_tlist, new_rt_index,
new_exclRelIndex, 0);
parsetree->onConflict = (OnConflictExpr *)
ReplaceVarsFromTargetList((Node *) parsetree->onConflict,
old_exclRelIndex,
0,
view_rte,
tmp_tlist,
REPLACEVARS_REPORT_ERROR,
0,
&parsetree->hasSubLinks);
}
/*
* For UPDATE/DELETE, pull up any WHERE quals from the view. We know that
* any Vars in the quals must reference the one base relation, so we need
* only adjust their varnos to reference the new target (just the same as
* we did with the view targetlist).
*
* If it's a security-barrier view, its WHERE quals must be applied before
* quals from the outer query, so we attach them to the RTE as security
* barrier quals rather than adding them to the main WHERE clause.
*
* For INSERT, the view's quals can be ignored in the main query.
*/
if (parsetree->commandType != CMD_INSERT &&
viewquery->jointree->quals != NULL)
{
Node *viewqual = (Node *) viewquery->jointree->quals;
Put back one copyObject() in rewriteTargetView(). Commit 6f8cb1e23485bd6d tried to centralize rewriteTargetView's copying of a target view's Query struct. However, it ignored the fact that the jointree->quals field was used twice. This only accidentally failed to fail immediately because the same ChangeVarNodes mutation is applied in both cases, so that we end up with logically identical expression trees for both uses (and, as the code stands, the second ChangeVarNodes call actually does nothing). However, we end up linking *physically* identical expression trees into both an RTE's securityQuals list and the WithCheckOption list. That's pretty dangerous, mainly because prepsecurity.c is utterly cavalier about further munging such structures without copying them first. There may be no live bug in HEAD as a consequence of the fact that we apply preprocess_expression in between here and prepsecurity.c, and that will make a copy of the tree anyway. Or it may just be that the regression tests happen to not trip over it. (I noticed this only because things fell over pretty badly when I tried to relocate the planner's call of expand_security_quals to before expression preprocessing.) In any case it's very fragile because if anyone tried to make the securityQuals and WithCheckOption trees diverge before we reach preprocess_expression, it would not work. The fact that the current code will preprocess securityQuals and WithCheckOptions lists at completely different times in different query levels does nothing to increase my trust that that can't happen. In view of the fact that 9.5.0 is almost upon us and the aforesaid commit has seen exactly zero field testing, the prudent course is to make an extra copy of the quals so that the behavior is not different from what has been in the field during beta.
2015-12-29 22:45:47 +01:00
/*
* Even though we copied viewquery already at the top of this
* function, we must duplicate the viewqual again here, because we may
* need to use the quals again below for a WithCheckOption clause.
*/
viewqual = copyObject(viewqual);
ChangeVarNodes(viewqual, base_rt_index, new_rt_index, 0);
if (RelationIsSecurityView(view))
{
/*
* The view's quals go in front of existing barrier quals: those
* would have come from an outer level of security-barrier view,
* and so must get evaluated later.
*
* Note: the parsetree has been mutated, so the new_rte pointer is
* stale and needs to be re-computed.
*/
new_rte = rt_fetch(new_rt_index, parsetree->rtable);
new_rte->securityQuals = lcons(viewqual, new_rte->securityQuals);
/*
* Do not set parsetree->hasRowSecurity, because these aren't RLS
* conditions (they aren't affected by enabling/disabling RLS).
*/
/*
* Make sure that the query is marked correctly if the added qual
* has sublinks.
*/
if (!parsetree->hasSubLinks)
parsetree->hasSubLinks = checkExprHasSubLink(viewqual);
}
else
AddQual(parsetree, (Node *) viewqual);
}
/*
* For INSERT/UPDATE, if the view has the WITH CHECK OPTION, or any parent
* view specified WITH CASCADED CHECK OPTION, add the quals from the view
* to the query's withCheckOptions list.
*/
if (parsetree->commandType != CMD_DELETE)
{
bool has_wco = RelationHasCheckOption(view);
bool cascaded = RelationHasCascadedCheckOption(view);
/*
* If the parent view has a cascaded check option, treat this view as
* if it also had a cascaded check option.
*
* New WithCheckOptions are added to the start of the list, so if
* there is a cascaded check option, it will be the first item in the
* list.
*/
if (parsetree->withCheckOptions != NIL)
{
WithCheckOption *parent_wco =
(WithCheckOption *) linitial(parsetree->withCheckOptions);
if (parent_wco->cascaded)
{
has_wco = true;
cascaded = true;
}
}
/*
* Add the new WithCheckOption to the start of the list, so that
* checks on inner views are run before checks on outer views, as
* required by the SQL standard.
*
* If the new check is CASCADED, we need to add it even if this view
* has no quals, since there may be quals on child views. A LOCAL
* check can be omitted if this view has no quals.
*/
if (has_wco && (cascaded || viewquery->jointree->quals != NULL))
{
WithCheckOption *wco;
wco = makeNode(WithCheckOption);
wco->kind = WCO_VIEW_CHECK;
wco->relname = pstrdup(RelationGetRelationName(view));
wco->polname = NULL;
wco->qual = NULL;
wco->cascaded = cascaded;
parsetree->withCheckOptions = lcons(wco,
parsetree->withCheckOptions);
if (viewquery->jointree->quals != NULL)
{
wco->qual = (Node *) viewquery->jointree->quals;
ChangeVarNodes(wco->qual, base_rt_index, new_rt_index, 0);
/*
* Make sure that the query is marked correctly if the added
* qual has sublinks. We can skip this check if the query is
* already marked, or if the command is an UPDATE, in which
* case the same qual will have already been added, and this
* check will already have been done.
*/
if (!parsetree->hasSubLinks &&
parsetree->commandType != CMD_UPDATE)
parsetree->hasSubLinks = checkExprHasSubLink(wco->qual);
}
}
}
table_close(base_rel, NoLock);
Fix INSERT ON CONFLICT UPDATE through a view that isn't just SELECT *. When expanding an updatable view that is an INSERT's target, the rewriter failed to rewrite Vars in the ON CONFLICT UPDATE clause. This accidentally worked if the view was just "SELECT * FROM ...", as the transformation would be a no-op in that case. With more complicated view targetlists, this omission would often lead to "attribute ... has the wrong type" errors or even crashes, as reported by Mario De Frutos Dieguez. Fix by adding code to rewriteTargetView to fix up the data structure correctly. The easiest way to update the exclRelTlist list is to rebuild it from scratch looking at the new target relation, so factor the code for that out of transformOnConflictClause to make it sharable. In passing, avoid duplicate permissions checks against the EXCLUDED pseudo-relation, and prevent useless view expansion of that relation's dummy RTE. The latter is only known to happen (after this patch) in cases where the query would fail later due to not having any INSTEAD OF triggers for the view. But by exactly that token, it would create an unintended and very poorly tested state of the query data structure, so it seems like a good idea to prevent it from happening at all. This has been broken since ON CONFLICT was introduced, so back-patch to 9.5. Dean Rasheed, based on an earlier patch by Amit Langote; comment-kibitzing and back-patching by me Discussion: https://postgr.es/m/CAFYwGJ0xfzy8jaK80hVN2eUWr6huce0RU8AgU04MGD00igqkTg@mail.gmail.com
2018-08-05 01:38:58 +02:00
return parsetree;
}
/*
* RewriteQuery -
* rewrites the query and apply the rules again on the queries rewritten
*
* rewrite_events is a list of open query-rewrite actions, so we can detect
* infinite recursion.
*
* orig_rt_length is the length of the originating query's rtable, for product
* queries created by fireRules(), and 0 otherwise. This is used to skip any
* already-processed VALUES RTEs from the original query.
*/
static List *
RewriteQuery(Query *parsetree, List *rewrite_events, int orig_rt_length)
{
CmdType event = parsetree->commandType;
bool instead = false;
bool returning = false;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
bool updatableview = false;
Query *qual_product = NULL;
List *rewritten = NIL;
ListCell *lc1;
/*
* First, recursively process any insert/update/delete statements in WITH
* clauses. (We have to do this first because the WITH clauses may get
* copied into rule actions below.)
*/
foreach(lc1, parsetree->cteList)
{
CommonTableExpr *cte = lfirst_node(CommonTableExpr, lc1);
2017-02-21 17:33:07 +01:00
Query *ctequery = castNode(Query, cte->ctequery);
List *newstuff;
if (ctequery->commandType == CMD_SELECT)
continue;
newstuff = RewriteQuery(ctequery, rewrite_events, 0);
/*
* Currently we can only handle unconditional, single-statement DO
* INSTEAD rules correctly; we have to get exactly one non-utility
* Query out of the rewrite operation to stuff back into the CTE node.
*/
if (list_length(newstuff) == 1)
{
/* Must check it's not a utility command */
ctequery = linitial_node(Query, newstuff);
if (!(ctequery->commandType == CMD_SELECT ||
ctequery->commandType == CMD_UPDATE ||
ctequery->commandType == CMD_INSERT ||
ctequery->commandType == CMD_DELETE))
{
/*
* Currently it could only be NOTIFY; this error message will
* need work if we ever allow other utility commands in rules.
*/
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("DO INSTEAD NOTIFY rules are not supported for data-modifying statements in WITH")));
}
/* WITH queries should never be canSetTag */
Assert(!ctequery->canSetTag);
/* Push the single Query back into the CTE node */
cte->ctequery = (Node *) ctequery;
}
else if (newstuff == NIL)
{
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("DO INSTEAD NOTHING rules are not supported for data-modifying statements in WITH")));
}
else
{
ListCell *lc2;
/* examine queries to determine which error message to issue */
foreach(lc2, newstuff)
{
Query *q = (Query *) lfirst(lc2);
if (q->querySource == QSRC_QUAL_INSTEAD_RULE)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("conditional DO INSTEAD rules are not supported for data-modifying statements in WITH")));
if (q->querySource == QSRC_NON_INSTEAD_RULE)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("DO ALSO rules are not supported for data-modifying statements in WITH")));
}
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("multi-statement DO INSTEAD rules are not supported for data-modifying statements in WITH")));
}
}
/*
* If the statement is an insert, update, or delete, adjust its targetlist
* as needed, and then fire INSERT/UPDATE/DELETE rules on it.
*
* SELECT rules are handled later when we have all the queries that should
* get executed. Also, utilities aren't rewritten at all (do we still
* need that check?)
*/
if (event != CMD_SELECT && event != CMD_UTILITY)
{
int result_relation;
RangeTblEntry *rt_entry;
Relation rt_entry_relation;
List *locks;
int product_orig_rt_length;
List *product_queries;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
bool hasUpdate = false;
int values_rte_index = 0;
bool defaults_remaining = false;
result_relation = parsetree->resultRelation;
Assert(result_relation != 0);
rt_entry = rt_fetch(result_relation, parsetree->rtable);
Assert(rt_entry->rtekind == RTE_RELATION);
/*
* We can use NoLock here since either the parser or
* AcquireRewriteLocks should have locked the rel already.
*/
rt_entry_relation = table_open(rt_entry->relid, NoLock);
/*
* Rewrite the targetlist as needed for the command type.
*/
if (event == CMD_INSERT)
{
ListCell *lc2;
RangeTblEntry *values_rte = NULL;
/*
* Test if it's a multi-row INSERT ... VALUES (...), (...), ... by
* looking for a VALUES RTE in the fromlist. For product queries,
* we must ignore any already-processed VALUES RTEs from the
* original query. These appear at the start of the rangetable.
*/
foreach(lc2, parsetree->jointree->fromlist)
{
RangeTblRef *rtr = (RangeTblRef *) lfirst(lc2);
if (IsA(rtr, RangeTblRef) && rtr->rtindex > orig_rt_length)
{
RangeTblEntry *rte = rt_fetch(rtr->rtindex,
parsetree->rtable);
if (rte->rtekind == RTE_VALUES)
{
/* should not find more than one VALUES RTE */
if (values_rte != NULL)
elog(ERROR, "more than one VALUES RTE found");
values_rte = rte;
values_rte_index = rtr->rtindex;
}
}
}
if (values_rte)
{
Bitmapset *unused_values_attrnos = NULL;
/* Process the main targetlist ... */
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
parsetree->targetList = rewriteTargetListIU(parsetree->targetList,
parsetree->commandType,
parsetree->override,
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
rt_entry_relation,
values_rte,
values_rte_index,
&unused_values_attrnos);
/* ... and the VALUES expression lists */
Further fixing for multi-row VALUES lists for updatable views. Previously, rewriteTargetListIU() generated a list of attribute numbers from the targetlist, which were passed to rewriteValuesRTE(), which expected them to contain the same number of entries as there are columns in the VALUES RTE, and to be in the same order. That was fine when the target relation was a table, but for an updatable view it could be broken in at least three different ways --- rewriteTargetListIU() could insert additional targetlist entries for view columns with defaults, the view columns could be in a different order from the columns of the underlying base relation, and targetlist entries could be merged together when assigning to elements of an array or composite type. As a result, when recursing to the base relation, the list of attribute numbers generated from the rewritten targetlist could no longer be relied upon to match the columns of the VALUES RTE. We got away with that prior to 41531e42d3 because it used to always be the case that rewriteValuesRTE() did nothing for the underlying base relation, since all DEFAULTS had already been replaced when it was initially invoked for the view, but that was incorrect because it failed to apply defaults from the base relation. Fix this by examining the targetlist entries more carefully and picking out just those that are simple Vars referencing the VALUES RTE. That's sufficient for the purposes of rewriteValuesRTE(), which is only responsible for dealing with DEFAULT items in the VALUES RTE. Any DEFAULT item in the VALUES RTE that doesn't have a matching simple-Var-assignment in the targetlist is an error which we complain about, but in theory that ought to be impossible. Additionally, move this code into rewriteValuesRTE() to give a clearer separation of concerns between the 2 functions. There is no need for rewriteTargetListIU() to know about the details of the VALUES RTE. While at it, fix the comment for rewriteValuesRTE() which claimed that it doesn't support array element and field assignments --- that hasn't been true since a3c7a993d5 (9.6 and later). Back-patch to all supported versions, with minor differences for the pre-9.6 branches, which don't support array element and field assignments to the same column in multi-row VALUES lists. Reviewed by Amit Langote. Discussion: https://postgr.es/m/15623-5d67a46788ec8b7f@postgresql.org
2019-03-03 11:51:13 +01:00
if (!rewriteValuesRTE(parsetree, values_rte, values_rte_index,
rt_entry_relation,
unused_values_attrnos))
defaults_remaining = true;
}
else
{
/* Process just the main targetlist */
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
parsetree->targetList =
rewriteTargetListIU(parsetree->targetList,
parsetree->commandType,
parsetree->override,
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
rt_entry_relation,
NULL, 0, NULL);
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
}
if (parsetree->onConflict &&
parsetree->onConflict->action == ONCONFLICT_UPDATE)
{
parsetree->onConflict->onConflictSet =
rewriteTargetListIU(parsetree->onConflict->onConflictSet,
CMD_UPDATE,
parsetree->override,
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
rt_entry_relation,
NULL, 0, NULL);
}
}
else if (event == CMD_UPDATE)
{
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
parsetree->targetList =
rewriteTargetListIU(parsetree->targetList,
parsetree->commandType,
parsetree->override,
rt_entry_relation,
NULL, 0, NULL);
}
else if (event == CMD_DELETE)
{
Fix creation of resjunk tlist entries for inherited mixed UPDATE/DELETE. rewriteTargetListUD's processing is dependent on the relkind of the query's target table. That was fine at the time it was made to act that way, even for queries on inheritance trees, because all tables in an inheritance tree would necessarily be plain tables. However, the 9.5 feature addition allowing some members of an inheritance tree to be foreign tables broke the assumption that rewriteTargetListUD's output tlist could be applied to all child tables with nothing more than column-number mapping. This led to visible failures if foreign child tables had row-level triggers, and would also break in cases where child tables belonged to FDWs that used methods other than CTID for row identification. To fix, delay running rewriteTargetListUD until after the planner has expanded inheritance, so that it is applied separately to the (already mapped) tlist for each child table. We can conveniently call it from preprocess_targetlist. Refactor associated code slightly to avoid the need to heap_open the target relation multiple times during preprocess_targetlist. (The APIs remain a bit ugly, particularly around the point of which steps scribble on parse->targetList and which don't. But avoiding such scribbling would require a change in FDW callback APIs, which is more pain than it's worth.) Also fix ExecModifyTable to ensure that "tupleid" is reset to NULL when we transition from rows providing a CTID to rows that don't. (That's really an independent bug, but it manifests in much the same cases.) Add a regression test checking one manifestation of this problem, which was that row-level triggers on a foreign child table did not work right. Back-patch to 9.5 where the problem was introduced. Etsuro Fujita, reviewed by Ildus Kurbangaliev and Ashutosh Bapat Discussion: https://postgr.es/m/20170514150525.0346ba72@postgrespro.ru
2017-11-27 23:53:56 +01:00
/* Nothing to do here */
}
else
elog(ERROR, "unrecognized commandType: %d", (int) event);
/*
* Collect and apply the appropriate rules.
*/
locks = matchLocks(event, rt_entry_relation->rd_rules,
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
result_relation, parsetree, &hasUpdate);
product_orig_rt_length = list_length(parsetree->rtable);
product_queries = fireRules(parsetree,
result_relation,
event,
locks,
&instead,
&returning,
&qual_product);
/*
* If we have a VALUES RTE with any remaining untouched DEFAULT items,
* and we got any product queries, finalize the VALUES RTE for each
* product query (replacing the remaining DEFAULT items with NULLs).
* We don't do this for the original query, because we know that it
* must be an auto-insert on a view, and so should use the base
* relation's defaults for any remaining DEFAULT items.
*/
if (defaults_remaining && product_queries != NIL)
{
ListCell *n;
/*
* Each product query has its own copy of the VALUES RTE at the
* same index in the rangetable, so we must finalize each one.
*
* Note that if the product query is an INSERT ... SELECT, then
* the VALUES RTE will be at the same index in the SELECT part of
* the product query rather than the top-level product query
* itself.
*/
foreach(n, product_queries)
{
Query *pt = (Query *) lfirst(n);
RangeTblEntry *values_rte;
if (pt->commandType == CMD_INSERT &&
pt->jointree && IsA(pt->jointree, FromExpr) &&
list_length(pt->jointree->fromlist) == 1)
{
Node *jtnode = (Node *) linitial(pt->jointree->fromlist);
if (IsA(jtnode, RangeTblRef))
{
int rtindex = ((RangeTblRef *) jtnode)->rtindex;
RangeTblEntry *src_rte = rt_fetch(rtindex, pt->rtable);
if (src_rte->rtekind == RTE_SUBQUERY &&
src_rte->subquery &&
IsA(src_rte->subquery, Query) &&
src_rte->subquery->commandType == CMD_SELECT)
pt = src_rte->subquery;
}
}
values_rte = rt_fetch(values_rte_index, pt->rtable);
if (values_rte->rtekind != RTE_VALUES)
elog(ERROR, "failed to find VALUES RTE in product query");
rewriteValuesRTEToNulls(pt, values_rte);
}
}
/*
* If there was no unqualified INSTEAD rule, and the target relation
* is a view without any INSTEAD OF triggers, see if the view can be
* automatically updated. If so, we perform the necessary query
* transformation here and add the resulting query to the
* product_queries list, so that it gets recursively rewritten if
* necessary.
*
* If the view cannot be automatically updated, we throw an error here
* which is OK since the query would fail at runtime anyway. Throwing
* the error here is preferable to the executor check since we have
* more detailed information available about why the view isn't
* updatable.
*/
if (!instead &&
rt_entry_relation->rd_rel->relkind == RELKIND_VIEW &&
!view_has_instead_trigger(rt_entry_relation, event))
{
/*
* If there were any qualified INSTEAD rules, don't allow the view
* to be automatically updated (an unqualified INSTEAD rule or
* INSTEAD OF trigger is required).
*
* The messages here should match execMain.c's CheckValidResultRel
* and in principle make those checks in executor unnecessary, but
* we keep them just in case.
*/
if (qual_product != NULL)
{
switch (parsetree->commandType)
{
case CMD_INSERT:
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("cannot insert into view \"%s\"",
RelationGetRelationName(rt_entry_relation)),
errdetail("Views with conditional DO INSTEAD rules are not automatically updatable."),
errhint("To enable inserting into the view, provide an INSTEAD OF INSERT trigger or an unconditional ON INSERT DO INSTEAD rule.")));
break;
case CMD_UPDATE:
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("cannot update view \"%s\"",
RelationGetRelationName(rt_entry_relation)),
errdetail("Views with conditional DO INSTEAD rules are not automatically updatable."),
errhint("To enable updating the view, provide an INSTEAD OF UPDATE trigger or an unconditional ON UPDATE DO INSTEAD rule.")));
break;
case CMD_DELETE:
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("cannot delete from view \"%s\"",
RelationGetRelationName(rt_entry_relation)),
errdetail("Views with conditional DO INSTEAD rules are not automatically updatable."),
errhint("To enable deleting from the view, provide an INSTEAD OF DELETE trigger or an unconditional ON DELETE DO INSTEAD rule.")));
break;
default:
elog(ERROR, "unrecognized CmdType: %d",
(int) parsetree->commandType);
break;
}
}
/*
* Attempt to rewrite the query to automatically update the view.
* This throws an error if the view can't be automatically
* updated.
*/
parsetree = rewriteTargetView(parsetree, rt_entry_relation);
/*
* At this point product_queries contains any DO ALSO rule
* actions. Add the rewritten query before or after those. This
* must match the handling the original query would have gotten
* below, if we allowed it to be included again.
*/
if (parsetree->commandType == CMD_INSERT)
product_queries = lcons(parsetree, product_queries);
else
product_queries = lappend(product_queries, parsetree);
/*
* Set the "instead" flag, as if there had been an unqualified
* INSTEAD, to prevent the original query from being included a
* second time below. The transformation will have rewritten any
* RETURNING list, so we can also set "returning" to forestall
* throwing an error below.
*/
instead = true;
returning = true;
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
updatableview = true;
}
/*
* If we got any product queries, recursively rewrite them --- but
* first check for recursion!
*/
if (product_queries != NIL)
{
ListCell *n;
rewrite_event *rev;
foreach(n, rewrite_events)
{
rev = (rewrite_event *) lfirst(n);
if (rev->relation == RelationGetRelid(rt_entry_relation) &&
rev->event == event)
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("infinite recursion detected in rules for relation \"%s\"",
RelationGetRelationName(rt_entry_relation))));
}
rev = (rewrite_event *) palloc(sizeof(rewrite_event));
rev->relation = RelationGetRelid(rt_entry_relation);
rev->event = event;
Avoid using lcons and list_delete_first where it's easy to do so. Formerly, lcons was about the same speed as lappend, but with the new List implementation, that's not so; with a long List, data movement imposes an O(N) cost on lcons and list_delete_first, but not lappend. Hence, invent list_delete_last with semantics parallel to list_delete_first (but O(1) cost), and change various places to use lappend and list_delete_last where this can be done without much violence to the code logic. There are quite a few places that construct result lists using lcons not lappend. Some have semantic rationales for that; I added comments about it to a couple that didn't have them already. In many such places though, I think the coding is that way only because back in the dark ages lcons was faster than lappend. Hence, switch to lappend where this can be done without causing semantic changes. In ExecInitExprRec(), this results in aggregates and window functions that are in the same plan node being executed in a different order than before. Generally, the executions of such functions ought to be independent of each other, so this shouldn't result in visibly different query results. But if you push it, as one regression test case does, you can show that the order is different. The new order seems saner; it's closer to the order of the functions in the query text. And we never documented or promised anything about this, anyway. Also, in gistfinishsplit(), don't bother building a reverse-order list; it's easy now to iterate backwards through the original list. It'd be possible to go further towards removing uses of lcons and list_delete_first, but it'd require more extensive logic changes, and I'm not convinced it's worth it. Most of the remaining uses deal with queues that probably never get long enough to be worth sweating over. (Actually, I doubt that any of the changes in this patch will have measurable performance effects either. But better to have good examples than bad ones in the code base.) Patch by me, thanks to David Rowley and Daniel Gustafsson for review. Discussion: https://postgr.es/m/21272.1563318411@sss.pgh.pa.us
2019-07-17 17:15:28 +02:00
rewrite_events = lappend(rewrite_events, rev);
foreach(n, product_queries)
{
Query *pt = (Query *) lfirst(n);
List *newstuff;
/*
* For an updatable view, pt might be the rewritten version of
* the original query, in which case we pass on orig_rt_length
* to finish processing any VALUES RTE it contained.
*
* Otherwise, we have a product query created by fireRules().
* Any VALUES RTEs from the original query have been fully
* processed, and must be skipped when we recurse.
*/
newstuff = RewriteQuery(pt, rewrite_events,
pt == parsetree ?
orig_rt_length :
product_orig_rt_length);
rewritten = list_concat(rewritten, newstuff);
}
Avoid using lcons and list_delete_first where it's easy to do so. Formerly, lcons was about the same speed as lappend, but with the new List implementation, that's not so; with a long List, data movement imposes an O(N) cost on lcons and list_delete_first, but not lappend. Hence, invent list_delete_last with semantics parallel to list_delete_first (but O(1) cost), and change various places to use lappend and list_delete_last where this can be done without much violence to the code logic. There are quite a few places that construct result lists using lcons not lappend. Some have semantic rationales for that; I added comments about it to a couple that didn't have them already. In many such places though, I think the coding is that way only because back in the dark ages lcons was faster than lappend. Hence, switch to lappend where this can be done without causing semantic changes. In ExecInitExprRec(), this results in aggregates and window functions that are in the same plan node being executed in a different order than before. Generally, the executions of such functions ought to be independent of each other, so this shouldn't result in visibly different query results. But if you push it, as one regression test case does, you can show that the order is different. The new order seems saner; it's closer to the order of the functions in the query text. And we never documented or promised anything about this, anyway. Also, in gistfinishsplit(), don't bother building a reverse-order list; it's easy now to iterate backwards through the original list. It'd be possible to go further towards removing uses of lcons and list_delete_first, but it'd require more extensive logic changes, and I'm not convinced it's worth it. Most of the remaining uses deal with queues that probably never get long enough to be worth sweating over. (Actually, I doubt that any of the changes in this patch will have measurable performance effects either. But better to have good examples than bad ones in the code base.) Patch by me, thanks to David Rowley and Daniel Gustafsson for review. Discussion: https://postgr.es/m/21272.1563318411@sss.pgh.pa.us
2019-07-17 17:15:28 +02:00
rewrite_events = list_delete_last(rewrite_events);
}
/*
* If there is an INSTEAD, and the original query has a RETURNING, we
* have to have found a RETURNING in the rule(s), else fail. (Because
* DefineQueryRewrite only allows RETURNING in unconditional INSTEAD
* rules, there's no need to worry whether the substituted RETURNING
* will actually be executed --- it must be.)
*/
if ((instead || qual_product != NULL) &&
parsetree->returningList &&
!returning)
{
switch (event)
{
case CMD_INSERT:
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot perform INSERT RETURNING on relation \"%s\"",
RelationGetRelationName(rt_entry_relation)),
errhint("You need an unconditional ON INSERT DO INSTEAD rule with a RETURNING clause.")));
break;
case CMD_UPDATE:
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot perform UPDATE RETURNING on relation \"%s\"",
RelationGetRelationName(rt_entry_relation)),
errhint("You need an unconditional ON UPDATE DO INSTEAD rule with a RETURNING clause.")));
break;
case CMD_DELETE:
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("cannot perform DELETE RETURNING on relation \"%s\"",
RelationGetRelationName(rt_entry_relation)),
errhint("You need an unconditional ON DELETE DO INSTEAD rule with a RETURNING clause.")));
break;
default:
elog(ERROR, "unrecognized commandType: %d",
(int) event);
break;
}
}
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
/*
* Updatable views are supported by ON CONFLICT, so don't prevent that
* case from proceeding
*/
if (parsetree->onConflict &&
(product_queries != NIL || hasUpdate) &&
!updatableview)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("INSERT with ON CONFLICT clause cannot be used with table that has INSERT or UPDATE rules")));
table_close(rt_entry_relation, NoLock);
}
/*
* For INSERTs, the original query is done first; for UPDATE/DELETE, it is
* done last. This is needed because update and delete rule actions might
* not do anything if they are invoked after the update or delete is
* performed. The command counter increment between the query executions
* makes the deleted (and maybe the updated) tuples disappear so the scans
* for them in the rule actions cannot find them.
*
* If we found any unqualified INSTEAD, the original query is not done at
* all, in any form. Otherwise, we add the modified form if qualified
* INSTEADs were found, else the unmodified form.
*/
if (!instead)
{
if (parsetree->commandType == CMD_INSERT)
{
if (qual_product != NULL)
rewritten = lcons(qual_product, rewritten);
else
rewritten = lcons(parsetree, rewritten);
}
else
{
if (qual_product != NULL)
rewritten = lappend(rewritten, qual_product);
else
rewritten = lappend(rewritten, parsetree);
}
}
/*
* If the original query has a CTE list, and we generated more than one
* non-utility result query, we have to fail because we'll have copied the
* CTE list into each result query. That would break the expectation of
* single evaluation of CTEs. This could possibly be fixed by
* restructuring so that a CTE list can be shared across multiple Query
* and PlannableStatement nodes.
*/
if (parsetree->cteList != NIL)
{
int qcount = 0;
foreach(lc1, rewritten)
{
Query *q = (Query *) lfirst(lc1);
if (q->commandType != CMD_UTILITY)
qcount++;
}
if (qcount > 1)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("WITH cannot be used in a query that is rewritten by rules into multiple queries")));
}
return rewritten;
}
/*
* QueryRewrite -
* Primary entry point to the query rewriter.
* Rewrite one query via query rewrite system, possibly returning 0
* or many queries.
*
* NOTE: the parsetree must either have come straight from the parser,
* or have been scanned by AcquireRewriteLocks to acquire suitable locks.
*/
List *
QueryRewrite(Query *parsetree)
{
uint64 input_query_id = parsetree->queryId;
List *querylist;
List *results;
ListCell *l;
CmdType origCmdType;
bool foundOriginalQuery;
Query *lastInstead;
/*
* This function is only applied to top-level original queries
*/
Assert(parsetree->querySource == QSRC_ORIGINAL);
Assert(parsetree->canSetTag);
/*
* Step 1
*
* Apply all non-SELECT rules possibly getting 0 or many queries
*/
querylist = RewriteQuery(parsetree, NIL, 0);
/*
* Step 2
*
* Apply all the RIR rules on each query
*
* This is also a handy place to mark each query with the original queryId
*/
results = NIL;
foreach(l, querylist)
{
Query *query = (Query *) lfirst(l);
1999-05-25 18:15:34 +02:00
query = fireRIRrules(query, NIL);
query->queryId = input_query_id;
results = lappend(results, query);
Hi! INTERSECT and EXCEPT is available for postgresql-v6.4! The patch against v6.4 is included at the end of the current text (in uuencoded form!) I also included the text of my Master's Thesis. (a postscript version). I hope that you find something of it useful and would be happy if parts of it find their way into the PostgreSQL documentation project (If so, tell me, then I send the sources of the document!) The contents of the document are: -) The first chapter might be of less interest as it gives only an overview on SQL. -) The second chapter gives a description on much of PostgreSQL's features (like user defined types etc. and how to use these features) -) The third chapter starts with an overview of PostgreSQL's internal structure with focus on the stages a query has to pass (i.e. parser, planner/optimizer, executor). Then a detailed description of the implementation of the Having clause and the Intersect/Except logic is given. Originally I worked on v6.3.2 but never found time enough to prepare and post a patch. Now I applied the changes to v6.4 to get Intersect and Except working with the new version. Chapter 3 of my documentation deals with the changes against v6.3.2, so keep that in mind when comparing the parts of the code printed there with the patched sources of v6.4. Here are some remarks on the patch. There are some things that have still to be done but at the moment I don't have time to do them myself. (I'm doing my military service at the moment) Sorry for that :-( -) I used a rewrite technique for the implementation of the Except/Intersect logic which rewrites the query to a semantically equivalent query before it is handed to the rewrite system (for views, rules etc.), planner, executor etc. -) In v6.3.2 the types of the attributes of two select statements connected by the UNION keyword had to match 100%. In v6.4 the types only need to be familiar (i.e. int and float can be mixed). Since this feature did not exist when I worked on Intersect/Except it does not work correctly for Except/Intersect queries WHEN USED IN COMBINATION WITH UNIONS! (i.e. sometimes the wrong type is used for the resulting table. This is because until now the types of the attributes of the first select statement have been used for the resulting table. When Intersects and/or Excepts are used in combination with Unions it might happen, that the first select statement of the original query appears at another position in the query which will be executed. The reason for this is the technique used for the implementation of Except/Intersect which does a query rewrite!) NOTE: It is NOT broken for pure UNION queries and pure INTERSECT/EXCEPT queries!!! -) I had to add the field intersect_clause to some data structures but did not find time to implement printfuncs for the new field. This does NOT break the debug modes but when an Except/Intersect is used the query debug output will be the already rewritten query. -) Massive changes to the grammar rules for SELECT and INSERT statements have been necessary (see comments in gram.y and documentation for deatails) in order to be able to use mixed queries like (SELECT ... UNION (SELECT ... EXCEPT SELECT)) INTERSECT SELECT...; -) When using UNION/EXCEPT/INTERSECT you will get: NOTICE: equal: "Don't know if nodes of type xxx are equal". I did not have time to add comparsion support for all the needed nodes, but the default behaviour of the function equal met my requirements. I did not dare to supress this message! That's the reason why the regression test for union will fail: These messages are also included in the union.out file! -) Somebody of you changed the union_planner() function for v6.4 (I copied the targetlist to new_tlist and that was removed and replaced by a cleanup of the original targetlist). These chnages violated some having queries executed against views so I changed it back again. I did not have time to examine the differences between the two versions but now it works :-) If you want to find out, try the file queries/view_having.sql on both versions and compare the results . Two queries won't produce a correct result with your version. regards Stefan
1999-01-18 01:10:17 +01:00
}
/*
* Step 3
*
* Determine which, if any, of the resulting queries is supposed to set
* the command-result tag; and update the canSetTag fields accordingly.
*
* If the original query is still in the list, it sets the command tag.
* Otherwise, the last INSTEAD query of the same kind as the original is
* allowed to set the tag. (Note these rules can leave us with no query
* setting the tag. The tcop code has to cope with this by setting up a
* default tag based on the original un-rewritten query.)
*
* The Asserts verify that at most one query in the result list is marked
* canSetTag. If we aren't checking asserts, we can fall out of the loop
* as soon as we find the original query.
*/
origCmdType = parsetree->commandType;
foundOriginalQuery = false;
lastInstead = NULL;
foreach(l, results)
{
Query *query = (Query *) lfirst(l);
if (query->querySource == QSRC_ORIGINAL)
{
Assert(query->canSetTag);
Assert(!foundOriginalQuery);
foundOriginalQuery = true;
#ifndef USE_ASSERT_CHECKING
break;
#endif
}
else
{
Assert(!query->canSetTag);
if (query->commandType == origCmdType &&
(query->querySource == QSRC_INSTEAD_RULE ||
query->querySource == QSRC_QUAL_INSTEAD_RULE))
lastInstead = query;
}
}
if (!foundOriginalQuery && lastInstead != NULL)
lastInstead->canSetTag = true;
return results;
Hi! INTERSECT and EXCEPT is available for postgresql-v6.4! The patch against v6.4 is included at the end of the current text (in uuencoded form!) I also included the text of my Master's Thesis. (a postscript version). I hope that you find something of it useful and would be happy if parts of it find their way into the PostgreSQL documentation project (If so, tell me, then I send the sources of the document!) The contents of the document are: -) The first chapter might be of less interest as it gives only an overview on SQL. -) The second chapter gives a description on much of PostgreSQL's features (like user defined types etc. and how to use these features) -) The third chapter starts with an overview of PostgreSQL's internal structure with focus on the stages a query has to pass (i.e. parser, planner/optimizer, executor). Then a detailed description of the implementation of the Having clause and the Intersect/Except logic is given. Originally I worked on v6.3.2 but never found time enough to prepare and post a patch. Now I applied the changes to v6.4 to get Intersect and Except working with the new version. Chapter 3 of my documentation deals with the changes against v6.3.2, so keep that in mind when comparing the parts of the code printed there with the patched sources of v6.4. Here are some remarks on the patch. There are some things that have still to be done but at the moment I don't have time to do them myself. (I'm doing my military service at the moment) Sorry for that :-( -) I used a rewrite technique for the implementation of the Except/Intersect logic which rewrites the query to a semantically equivalent query before it is handed to the rewrite system (for views, rules etc.), planner, executor etc. -) In v6.3.2 the types of the attributes of two select statements connected by the UNION keyword had to match 100%. In v6.4 the types only need to be familiar (i.e. int and float can be mixed). Since this feature did not exist when I worked on Intersect/Except it does not work correctly for Except/Intersect queries WHEN USED IN COMBINATION WITH UNIONS! (i.e. sometimes the wrong type is used for the resulting table. This is because until now the types of the attributes of the first select statement have been used for the resulting table. When Intersects and/or Excepts are used in combination with Unions it might happen, that the first select statement of the original query appears at another position in the query which will be executed. The reason for this is the technique used for the implementation of Except/Intersect which does a query rewrite!) NOTE: It is NOT broken for pure UNION queries and pure INTERSECT/EXCEPT queries!!! -) I had to add the field intersect_clause to some data structures but did not find time to implement printfuncs for the new field. This does NOT break the debug modes but when an Except/Intersect is used the query debug output will be the already rewritten query. -) Massive changes to the grammar rules for SELECT and INSERT statements have been necessary (see comments in gram.y and documentation for deatails) in order to be able to use mixed queries like (SELECT ... UNION (SELECT ... EXCEPT SELECT)) INTERSECT SELECT...; -) When using UNION/EXCEPT/INTERSECT you will get: NOTICE: equal: "Don't know if nodes of type xxx are equal". I did not have time to add comparsion support for all the needed nodes, but the default behaviour of the function equal met my requirements. I did not dare to supress this message! That's the reason why the regression test for union will fail: These messages are also included in the union.out file! -) Somebody of you changed the union_planner() function for v6.4 (I copied the targetlist to new_tlist and that was removed and replaced by a cleanup of the original targetlist). These chnages violated some having queries executed against views so I changed it back again. I did not have time to examine the differences between the two versions but now it works :-) If you want to find out, try the file queries/view_having.sql on both versions and compare the results . Two queries won't produce a correct result with your version. regards Stefan
1999-01-18 01:10:17 +01:00
}