2010-09-20 22:08:53 +02:00
<!-- doc/src/sgml/queries.sgml -->
2001-01-23 00:34:33 +01:00
<chapter id="queries">
<title>Queries</title>
2003-08-31 19:32:24 +02:00
<indexterm zone="queries">
<primary>query</primary>
</indexterm>
<indexterm zone="queries">
<primary>SELECT</primary>
</indexterm>
2002-08-05 21:43:31 +02:00
<para>
The previous chapters explained how to create tables, how to fill
them with data, and how to manipulate that data. Now we finally
2009-04-27 18:27:36 +02:00
discuss how to retrieve the data from the database.
2002-08-05 21:43:31 +02:00
</para>
2002-09-20 20:39:41 +02:00
2001-11-28 21:49:10 +01:00
<sect1 id="queries-overview">
<title>Overview</title>
2002-09-20 20:39:41 +02:00
<para>
The process of retrieving or the command to retrieve data from a
database is called a <firstterm>query</firstterm>. In SQL the
Improve <xref> vs. <command> formatting in the documentation
SQL commands are generally marked up as <command>, except when a link
to a reference page is used using <xref>. But the latter doesn't
create monospace markup, so this looks strange especially when a
paragraph contains a mix of links and non-links.
We considered putting <command> in the <refentrytitle> on the target
side, but that creates some formatting side effects elsewhere.
Generally, it seems safer to solve this on the link source side.
We can't put the <xref> inside the <command>; the DTD doesn't allow
this. DocBook 5 would allow the <command> to have the linkend
attribute itself, but we are not there yet.
So to solve this for now, convert the <xref>s to <link> plus
<command>. This gives the correct look and also gives some more
flexibility what we can put into the link text (e.g., subcommands or
other clauses). In the future, these could then be converted to
DocBook 5 style.
I haven't converted absolutely all xrefs to SQL command reference
pages, only those where we care about the appearance of the link text
or where it was otherwise appropriate to make the appearance match a
bit better. Also in some cases, the links where repetitive, so in
those cases the links where just removed and replaced by a plain
<command>. In cases where we just want the link and don't
specifically care about the generated link text (typically phrased
"for further information see <xref ...>") the xref is kept.
Reported-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/87o8pco34z.fsf@wibble.ilmari.org
2020-10-03 16:16:51 +02:00
<link linkend="sql-select"><command>SELECT</command></link> command is
2006-02-19 00:14:45 +01:00
used to specify queries. The general syntax of the
<command>SELECT</command> command is
2001-01-23 00:34:33 +01:00
<synopsis>
2008-10-04 23:56:55 +02:00
<optional>WITH <replaceable>with_queries</replaceable></optional> SELECT <replaceable>select_list</replaceable> FROM <replaceable>table_expression</replaceable> <optional><replaceable>sort_specification</replaceable></optional>
2001-01-23 00:34:33 +01:00
</synopsis>
2002-09-20 20:39:41 +02:00
The following sections describe the details of the select list, the
2017-10-09 03:44:17 +02:00
table expression, and the sort specification. <literal>WITH</literal>
2008-10-04 23:56:55 +02:00
queries are treated last since they are an advanced feature.
2002-09-20 20:39:41 +02:00
</para>
<para>
2007-02-01 01:28:19 +01:00
A simple kind of query has the form:
2001-01-23 00:34:33 +01:00
<programlisting>
SELECT * FROM table1;
</programlisting>
2002-09-20 20:39:41 +02:00
Assuming that there is a table called <literal>table1</literal>,
2011-09-10 16:45:55 +02:00
this command would retrieve all rows and all user-defined columns from
2002-09-20 20:39:41 +02:00
<literal>table1</literal>. (The method of retrieval depends on the
client application. For example, the
2001-01-23 00:34:33 +01:00
<application>psql</application> program will display an ASCII-art
2002-10-20 07:05:46 +02:00
table on the screen, while client libraries will offer functions to
2003-11-06 23:21:47 +01:00
extract individual values from the query result.) The select list
2001-01-23 00:34:33 +01:00
specification <literal>*</literal> means all columns that the table
expression happens to provide. A select list can also select a
2002-10-20 07:05:46 +02:00
subset of the available columns or make calculations using the
columns. For example, if
2017-10-09 03:44:17 +02:00
<literal>table1</literal> has columns named <literal>a</literal>,
<literal>b</literal>, and <literal>c</literal> (and perhaps others) you can make
2002-09-20 20:39:41 +02:00
the following query:
2001-01-23 00:34:33 +01:00
<programlisting>
SELECT a, b + c FROM table1;
</programlisting>
2017-10-09 03:44:17 +02:00
(assuming that <literal>b</literal> and <literal>c</literal> are of a numerical
2002-09-20 20:39:41 +02:00
data type).
2017-11-23 15:39:47 +01:00
See <xref linkend="queries-select-lists"/> for more details.
2001-01-23 00:34:33 +01:00
</para>
<para>
2009-04-27 18:27:36 +02:00
<literal>FROM table1</literal> is a simple kind of
2002-09-20 20:39:41 +02:00
table expression: it reads just one table. In general, table
expressions can be complex constructs of base tables, joins, and
subqueries. But you can also omit the table expression entirely and
use the <command>SELECT</command> command as a calculator:
2001-01-23 00:34:33 +01:00
<programlisting>
SELECT 3 * 4;
</programlisting>
This is more useful if the expressions in the select list return
2002-10-20 07:05:46 +02:00
varying results. For example, you could call a function this way:
2001-01-23 00:34:33 +01:00
<programlisting>
SELECT random();
</programlisting>
2002-09-20 20:39:41 +02:00
</para>
2001-11-28 21:49:10 +01:00
</sect1>
2001-01-23 00:34:33 +01:00
2002-09-20 20:39:41 +02:00
2001-01-23 00:34:33 +01:00
<sect1 id="queries-table-expressions">
<title>Table Expressions</title>
2003-08-31 19:32:24 +02:00
<indexterm zone="queries-table-expressions">
<primary>table expression</primary>
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
2002-09-20 20:39:41 +02:00
A <firstterm>table expression</firstterm> computes a table. The
2017-10-09 03:44:17 +02:00
table expression contains a <literal>FROM</literal> clause that is
optionally followed by <literal>WHERE</literal>, <literal>GROUP BY</literal>, and
<literal>HAVING</literal> clauses. Trivial table expressions simply refer
2002-09-20 20:39:41 +02:00
to a table on disk, a so-called base table, but more complex
expressions can be used to modify or combine base tables in various
ways.
2001-01-23 00:34:33 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
The optional <literal>WHERE</literal>, <literal>GROUP BY</literal>, and
<literal>HAVING</literal> clauses in the table expression specify a
2002-09-20 20:39:41 +02:00
pipeline of successive transformations performed on the table
2017-10-09 03:44:17 +02:00
derived in the <literal>FROM</literal> clause. All these transformations
2002-09-20 20:39:41 +02:00
produce a virtual table that provides the rows that are passed to
the select list to compute the output rows of the query.
2001-01-23 00:34:33 +01:00
</para>
2006-09-18 21:54:01 +02:00
2001-01-23 00:34:33 +01:00
<sect2 id="queries-from">
2003-11-04 10:55:39 +01:00
<title>The <literal>FROM</literal> Clause</title>
2008-10-04 23:56:55 +02:00
2001-01-23 00:34:33 +01:00
<para>
Doc: fix "Unresolved ID reference" warnings, clean up man page cross-refs.
Use xreflabel attributes instead of endterm attributes to control the
appearance of links to subsections of SQL command reference pages.
This is simpler, it matches what we do elsewhere (e.g. for GUC variables),
and it doesn't draw "Unresolved ID reference" warnings from the PDF
toolchain.
Fix some places where the text was absolutely dependent on an <xref>
rendering exactly so, by using a <link> around the required text
instead. At least one of those spots had already been turned into
bad grammar by subsequent changes, and the whole idea is just too
fragile for my taste. <xref> does NOT have fixed output, don't write
as if it does.
Consistently include a page-level link in cross-man-page references,
because otherwise they are useless/nonsensical in man-page output.
Likewise, be consistent about mentioning "below" or "above" in same-page
references; we were doing that in about 90% of the cases, but now it's
100%.
Also get rid of another nonfunctional-in-PDF idea, of making
cross-references to functions by sticking ID tags on <row> constructs.
We can put the IDs on <indexterm>s instead --- which is probably not any
more sensible in abstract terms, but it works where the other doesn't.
(There is talk of attaching cross-reference IDs to most or all of
the docs' function descriptions, but for now I just fixed the two
that exist.)
Discussion: https://postgr.es/m/14480.1589154358@sss.pgh.pa.us
2020-05-11 20:15:49 +02:00
The <link linkend="sql-from"><literal>FROM</literal></link> clause derives a
2004-03-03 23:22:24 +01:00
table from one or more other tables given in a comma-separated
table reference list.
2001-01-23 00:34:33 +01:00
<synopsis>
FROM <replaceable>table_reference</replaceable> <optional>, <replaceable>table_reference</replaceable> <optional>, ...</optional></optional>
</synopsis>
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
A table reference can be a table name (possibly schema-qualified),
2017-10-09 03:44:17 +02:00
or a derived table such as a subquery, a <literal>JOIN</literal> construct, or
2014-11-19 22:00:24 +01:00
complex combinations of these. If more than one table reference is
2017-10-09 03:44:17 +02:00
listed in the <literal>FROM</literal> clause, the tables are cross-joined
2014-11-19 22:00:24 +01:00
(that is, the Cartesian product of their rows is formed; see below).
2017-10-09 03:44:17 +02:00
The result of the <literal>FROM</literal> list is an intermediate virtual
2014-11-19 22:00:24 +01:00
table that can then be subject to
2017-10-09 03:44:17 +02:00
transformations by the <literal>WHERE</literal>, <literal>GROUP BY</literal>,
and <literal>HAVING</literal> clauses and is finally the result of the
2002-09-20 20:39:41 +02:00
overall table expression.
2001-01-23 00:34:33 +01:00
</para>
2003-08-31 19:32:24 +02:00
<indexterm>
<primary>ONLY</primary>
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
2006-10-22 05:03:41 +02:00
When a table reference names a table that is the parent of a
2009-06-17 23:58:49 +02:00
table inheritance hierarchy, the table reference produces rows of
not only that table but all of its descendant tables, unless the
2017-10-09 03:44:17 +02:00
key word <literal>ONLY</literal> precedes the table name. However, the
2002-09-20 20:39:41 +02:00
reference produces only the columns that appear in the named table
2004-11-15 07:32:15 +01:00
— any columns added in subtables are ignored.
2001-01-23 00:34:33 +01:00
</para>
2012-09-17 20:59:31 +02:00
<para>
2017-10-09 03:44:17 +02:00
Instead of writing <literal>ONLY</literal> before the table name, you can write
<literal>*</literal> after the table name to explicitly specify that descendant
2016-12-23 13:35:01 +01:00
tables are included. There is no real reason to use this syntax any more,
2016-12-23 17:53:35 +01:00
because searching descendant tables is now always the default behavior.
2016-12-23 13:35:01 +01:00
However, it is supported for compatibility with older releases.
2012-09-17 20:59:31 +02:00
</para>
2001-01-23 00:34:33 +01:00
<sect3 id="queries-join">
<title>Joined Tables</title>
2001-05-13 00:51:36 +02:00
<indexterm zone="queries-join">
2003-08-31 19:32:24 +02:00
<primary>join</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
A joined table is a table derived from two other (real or
derived) tables according to the rules of the particular join
2002-08-05 21:43:31 +02:00
type. Inner, outer, and cross-joins are available.
2014-11-19 22:00:24 +01:00
The general syntax of a joined table is
<synopsis>
<replaceable>T1</replaceable> <replaceable>join_type</replaceable> <replaceable>T2</replaceable> <optional> <replaceable>join_condition</replaceable> </optional>
</synopsis>
Joins of all types can be chained together, or nested: either or
both <replaceable>T1</replaceable> and
<replaceable>T2</replaceable> can be joined tables. Parentheses
2017-10-09 03:44:17 +02:00
can be used around <literal>JOIN</literal> clauses to control the join
order. In the absence of parentheses, <literal>JOIN</literal> clauses
2014-11-19 22:00:24 +01:00
nest left-to-right.
2001-01-23 00:34:33 +01:00
</para>
<variablelist>
<title>Join Types</title>
<varlistentry>
2014-05-07 03:28:58 +02:00
<term>Cross join
2001-11-09 00:40:40 +01:00
<indexterm>
2003-08-31 19:32:24 +02:00
<primary>join</primary>
2001-11-09 00:40:40 +01:00
<secondary>cross</secondary>
</indexterm>
2003-08-31 19:32:24 +02:00
<indexterm>
<primary>cross join</primary>
</indexterm>
2014-05-07 03:28:58 +02:00
</term>
2003-08-31 19:32:24 +02:00
2001-01-23 00:34:33 +01:00
<listitem>
<synopsis>
<replaceable>T1</replaceable> CROSS JOIN <replaceable>T2</replaceable>
</synopsis>
<para>
2009-06-17 23:58:49 +02:00
For every possible combination of rows from
2001-01-23 00:34:33 +01:00
<replaceable>T1</replaceable> and
2009-04-27 18:27:36 +02:00
<replaceable>T2</replaceable> (i.e., a Cartesian product),
2009-06-17 23:58:49 +02:00
the joined table will contain a
row consisting of all columns in <replaceable>T1</replaceable>
followed by all columns in <replaceable>T2</replaceable>. If
2001-02-15 05:10:54 +01:00
the tables have N and M rows respectively, the joined
2003-03-13 02:30:29 +01:00
table will have N * M rows.
2001-01-23 00:34:33 +01:00
</para>
2003-03-13 02:30:29 +01:00
<para>
<literal>FROM <replaceable>T1</replaceable> CROSS JOIN
<replaceable>T2</replaceable></literal> is equivalent to
<literal>FROM <replaceable>T1</replaceable> INNER JOIN
<replaceable>T2</replaceable> ON TRUE</literal> (see below).
2014-11-19 22:00:24 +01:00
It is also equivalent to
<literal>FROM <replaceable>T1</replaceable>,
<replaceable>T2</replaceable></literal>.
<note>
<para>
This latter equivalence does not hold exactly when more than two
2017-10-09 03:44:17 +02:00
tables appear, because <literal>JOIN</literal> binds more tightly than
2014-11-19 22:00:24 +01:00
comma. For example
<literal>FROM <replaceable>T1</replaceable> CROSS JOIN
<replaceable>T2</replaceable> INNER JOIN <replaceable>T3</replaceable>
ON <replaceable>condition</replaceable></literal>
is not the same as
<literal>FROM <replaceable>T1</replaceable>,
<replaceable>T2</replaceable> INNER JOIN <replaceable>T3</replaceable>
ON <replaceable>condition</replaceable></literal>
because the <replaceable>condition</replaceable> can
reference <replaceable>T1</replaceable> in the first case but not
the second.
</para>
</note>
2003-03-13 02:30:29 +01:00
</para>
2001-01-23 00:34:33 +01:00
</listitem>
</varlistentry>
<varlistentry>
2014-05-07 03:28:58 +02:00
<term>Qualified joins
2001-11-09 00:40:40 +01:00
<indexterm>
2003-08-31 19:32:24 +02:00
<primary>join</primary>
2001-11-09 00:40:40 +01:00
<secondary>outer</secondary>
</indexterm>
2001-05-13 00:51:36 +02:00
2003-08-31 19:32:24 +02:00
<indexterm>
<primary>outer join</primary>
</indexterm>
2014-05-07 03:28:58 +02:00
</term>
2003-08-31 19:32:24 +02:00
2001-11-09 00:40:40 +01:00
<listitem>
2001-01-23 00:34:33 +01:00
<synopsis>
2001-03-25 00:03:26 +01:00
<replaceable>T1</replaceable> { <optional>INNER</optional> | { LEFT | RIGHT | FULL } <optional>OUTER</optional> } JOIN <replaceable>T2</replaceable> ON <replaceable>boolean_expression</replaceable>
2001-01-23 00:34:33 +01:00
<replaceable>T1</replaceable> { <optional>INNER</optional> | { LEFT | RIGHT | FULL } <optional>OUTER</optional> } JOIN <replaceable>T2</replaceable> USING ( <replaceable>join column list</replaceable> )
2001-02-15 05:10:54 +01:00
<replaceable>T1</replaceable> NATURAL { <optional>INNER</optional> | { LEFT | RIGHT | FULL } <optional>OUTER</optional> } JOIN <replaceable>T2</replaceable>
2001-01-23 00:34:33 +01:00
</synopsis>
2008-10-04 23:56:55 +02:00
2001-01-23 00:34:33 +01:00
<para>
2002-09-20 20:39:41 +02:00
The words <literal>INNER</literal> and
<literal>OUTER</literal> are optional in all forms.
<literal>INNER</literal> is the default;
<literal>LEFT</literal>, <literal>RIGHT</literal>, and
<literal>FULL</literal> imply an outer join.
2001-01-23 00:34:33 +01:00
</para>
<para>
The <firstterm>join condition</firstterm> is specified in the
2017-10-09 03:44:17 +02:00
<literal>ON</literal> or <literal>USING</literal> clause, or implicitly by
the word <literal>NATURAL</literal>. The join condition determines
2002-09-20 20:39:41 +02:00
which rows from the two source tables are considered to
<quote>match</quote>, as explained in detail below.
2001-02-15 05:10:54 +01:00
</para>
<para>
2002-09-20 20:39:41 +02:00
The possible types of qualified join are:
2001-02-15 05:10:54 +01:00
2001-01-23 00:34:33 +01:00
<variablelist>
<varlistentry>
2017-10-09 03:44:17 +02:00
<term><literal>INNER JOIN</literal></term>
2001-01-23 00:34:33 +01:00
<listitem>
<para>
For each row R1 of T1, the joined table has a row for each
row in T2 that satisfies the join condition with R1.
</para>
</listitem>
</varlistentry>
<varlistentry>
2017-10-09 03:44:17 +02:00
<term><literal>LEFT OUTER JOIN</literal>
2001-11-09 00:40:40 +01:00
<indexterm>
2003-08-31 19:32:24 +02:00
<primary>join</primary>
2001-11-09 00:40:40 +01:00
<secondary>left</secondary>
</indexterm>
2001-05-13 00:51:36 +02:00
2003-08-31 19:32:24 +02:00
<indexterm>
<primary>left join</primary>
</indexterm>
2014-05-07 03:28:58 +02:00
</term>
2003-08-31 19:32:24 +02:00
2001-11-09 00:40:40 +01:00
<listitem>
2001-01-23 00:34:33 +01:00
<para>
2002-09-20 20:39:41 +02:00
First, an inner join is performed. Then, for each row in
T1 that does not satisfy the join condition with any row in
2009-06-17 23:58:49 +02:00
T2, a joined row is added with null values in columns of
2009-04-27 18:27:36 +02:00
T2. Thus, the joined table always has at least
2002-09-20 20:39:41 +02:00
one row for each row in T1.
2001-01-23 00:34:33 +01:00
</para>
</listitem>
</varlistentry>
2008-10-04 23:56:55 +02:00
2001-01-23 00:34:33 +01:00
<varlistentry>
2017-10-09 03:44:17 +02:00
<term><literal>RIGHT OUTER JOIN</literal>
2003-08-31 19:32:24 +02:00
<indexterm>
<primary>join</primary>
<secondary>right</secondary>
</indexterm>
<indexterm>
<primary>right join</primary>
</indexterm>
2014-05-07 03:28:58 +02:00
</term>
2003-08-31 19:32:24 +02:00
2001-01-23 00:34:33 +01:00
<listitem>
<para>
2002-09-20 20:39:41 +02:00
First, an inner join is performed. Then, for each row in
T2 that does not satisfy the join condition with any row in
2009-06-17 23:58:49 +02:00
T1, a joined row is added with null values in columns of
2002-09-20 20:39:41 +02:00
T1. This is the converse of a left join: the result table
2009-04-27 18:27:36 +02:00
will always have a row for each row in T2.
2001-01-23 00:34:33 +01:00
</para>
</listitem>
</varlistentry>
2008-10-04 23:56:55 +02:00
2001-01-23 00:34:33 +01:00
<varlistentry>
2017-10-09 03:44:17 +02:00
<term><literal>FULL OUTER JOIN</literal></term>
2001-01-23 00:34:33 +01:00
<listitem>
<para>
2002-09-20 20:39:41 +02:00
First, an inner join is performed. Then, for each row in
2001-01-23 00:34:33 +01:00
T1 that does not satisfy the join condition with any row in
2009-06-17 23:58:49 +02:00
T2, a joined row is added with null values in columns of
2001-01-23 00:34:33 +01:00
T2. Also, for each row of T2 that does not satisfy the
2009-06-17 23:58:49 +02:00
join condition with any row in T1, a joined row with null
2002-10-20 07:05:46 +02:00
values in the columns of T1 is added.
2001-01-23 00:34:33 +01:00
</para>
</listitem>
</varlistentry>
</variablelist>
2003-03-13 02:30:29 +01:00
</para>
2014-11-19 22:00:24 +01:00
<para>
2017-10-09 03:44:17 +02:00
The <literal>ON</literal> clause is the most general kind of join
2014-11-19 22:00:24 +01:00
condition: it takes a Boolean value expression of the same
2017-10-09 03:44:17 +02:00
kind as is used in a <literal>WHERE</literal> clause. A pair of rows
from <replaceable>T1</replaceable> and <replaceable>T2</replaceable> match if the
<literal>ON</literal> expression evaluates to true.
2014-11-19 22:00:24 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
The <literal>USING</literal> clause is a shorthand that allows you to take
2014-11-19 22:00:24 +01:00
advantage of the specific situation where both sides of the join use
the same name for the joining column(s). It takes a
comma-separated list of the shared column names
and forms a join condition that includes an equality comparison
2017-10-09 03:44:17 +02:00
for each one. For example, joining <replaceable>T1</replaceable>
and <replaceable>T2</replaceable> with <literal>USING (a, b)</literal> produces
the join condition <literal>ON <replaceable>T1</replaceable>.a
= <replaceable>T2</replaceable>.a AND <replaceable>T1</replaceable>.b
= <replaceable>T2</replaceable>.b</literal>.
2014-11-19 22:00:24 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
Furthermore, the output of <literal>JOIN USING</literal> suppresses
2014-11-19 22:00:24 +01:00
redundant columns: there is no need to print both of the matched
columns, since they must have equal values. While <literal>JOIN
2017-10-09 03:44:17 +02:00
ON</literal> produces all columns from <replaceable>T1</replaceable> followed by all
columns from <replaceable>T2</replaceable>, <literal>JOIN USING</literal> produces one
2014-11-19 22:00:24 +01:00
output column for each of the listed column pairs (in the listed
2017-10-09 03:44:17 +02:00
order), followed by any remaining columns from <replaceable>T1</replaceable>,
followed by any remaining columns from <replaceable>T2</replaceable>.
2014-11-19 22:00:24 +01:00
</para>
<para>
<indexterm>
<primary>join</primary>
<secondary>natural</secondary>
</indexterm>
<indexterm>
<primary>natural join</primary>
</indexterm>
2017-10-09 03:44:17 +02:00
Finally, <literal>NATURAL</literal> is a shorthand form of
<literal>USING</literal>: it forms a <literal>USING</literal> list
2014-11-19 22:00:24 +01:00
consisting of all column names that appear in both
2017-10-09 03:44:17 +02:00
input tables. As with <literal>USING</literal>, these columns appear
2014-11-19 22:00:24 +01:00
only once in the output table. If there are no common
2017-07-20 18:41:26 +02:00
column names, <literal>NATURAL JOIN</literal> behaves like
<literal>JOIN ... ON TRUE</literal>, producing a cross-product join.
2014-11-19 22:00:24 +01:00
</para>
<note>
<para>
<literal>USING</literal> is reasonably safe from column changes
in the joined relations since only the listed columns
2017-10-09 03:44:17 +02:00
are combined. <literal>NATURAL</literal> is considerably more risky since
2014-11-19 22:00:24 +01:00
any schema changes to either relation that cause a new matching
column name to be present will cause the join to combine that new
column as well.
</para>
</note>
2001-01-23 00:34:33 +01:00
</listitem>
</varlistentry>
</variablelist>
<para>
2007-02-01 01:28:19 +01:00
To put this together, assume we have tables <literal>t1</literal>:
2001-01-23 00:34:33 +01:00
<programlisting>
2002-09-20 20:39:41 +02:00
num | name
-----+------
1 | a
2 | b
3 | c
2001-01-23 00:34:33 +01:00
</programlisting>
2007-02-01 01:28:19 +01:00
and <literal>t2</literal>:
2002-09-20 20:39:41 +02:00
<programlisting>
num | value
-----+-------
1 | xxx
3 | yyy
5 | zzz
</programlisting>
then we get the following results for the various joins:
<screen>
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM t1 CROSS JOIN t2;</userinput>
2002-09-20 20:39:41 +02:00
num | name | num | value
-----+------+-----+-------
1 | a | 1 | xxx
1 | a | 3 | yyy
1 | a | 5 | zzz
2 | b | 1 | xxx
2 | b | 3 | yyy
2 | b | 5 | zzz
3 | c | 1 | xxx
3 | c | 3 | yyy
3 | c | 5 | zzz
(9 rows)
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM t1 INNER JOIN t2 ON t1.num = t2.num;</userinput>
2002-09-20 20:39:41 +02:00
num | name | num | value
-----+------+-----+-------
1 | a | 1 | xxx
3 | c | 3 | yyy
(2 rows)
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM t1 INNER JOIN t2 USING (num);</userinput>
2002-09-20 20:39:41 +02:00
num | name | value
-----+------+-------
1 | a | xxx
3 | c | yyy
(2 rows)
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM t1 NATURAL INNER JOIN t2;</userinput>
2002-09-20 20:39:41 +02:00
num | name | value
-----+------+-------
1 | a | xxx
3 | c | yyy
(2 rows)
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num;</userinput>
2002-09-20 20:39:41 +02:00
num | name | num | value
-----+------+-----+-------
1 | a | 1 | xxx
2 | b | |
3 | c | 3 | yyy
(3 rows)
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM t1 LEFT JOIN t2 USING (num);</userinput>
2002-09-20 20:39:41 +02:00
num | name | value
-----+------+-------
1 | a | xxx
2 | b |
3 | c | yyy
(3 rows)
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM t1 RIGHT JOIN t2 ON t1.num = t2.num;</userinput>
2002-09-20 20:39:41 +02:00
num | name | num | value
-----+------+-----+-------
1 | a | 1 | xxx
3 | c | 3 | yyy
| | 5 | zzz
(3 rows)
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM t1 FULL JOIN t2 ON t1.num = t2.num;</userinput>
2002-09-20 20:39:41 +02:00
num | name | num | value
-----+------+-----+-------
1 | a | 1 | xxx
2 | b | |
3 | c | 3 | yyy
| | 5 | zzz
(4 rows)
</screen>
</para>
2001-01-23 00:34:33 +01:00
<para>
2017-10-09 03:44:17 +02:00
The join condition specified with <literal>ON</literal> can also contain
2002-09-20 20:39:41 +02:00
conditions that do not relate directly to the join. This can
prove useful for some queries but needs to be thought out
carefully. For example:
<screen>
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num AND t2.value = 'xxx';</userinput>
2002-09-20 20:39:41 +02:00
num | name | num | value
-----+------+-----+-------
1 | a | 1 | xxx
2 | b | |
3 | c | |
(3 rows)
</screen>
2017-10-09 03:44:17 +02:00
Notice that placing the restriction in the <literal>WHERE</literal> clause
2009-04-27 18:27:36 +02:00
produces a different result:
<screen>
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM t1 LEFT JOIN t2 ON t1.num = t2.num WHERE t2.value = 'xxx';</userinput>
2009-04-27 18:27:36 +02:00
num | name | num | value
-----+------+-----+-------
1 | a | 1 | xxx
(1 row)
</screen>
2017-10-09 03:44:17 +02:00
This is because a restriction placed in the <literal>ON</literal>
clause is processed <emphasis>before</emphasis> the join, while
a restriction placed in the <literal>WHERE</literal> clause is processed
<emphasis>after</emphasis> the join.
2014-11-19 22:00:24 +01:00
That does not matter with inner joins, but it matters a lot with outer
joins.
2001-01-23 00:34:33 +01:00
</para>
</sect3>
<sect3 id="queries-table-aliases">
<title>Table and Column Aliases</title>
2001-05-13 00:51:36 +02:00
<indexterm zone="queries-table-aliases">
2003-08-31 19:32:24 +02:00
<primary>alias</primary>
<secondary>in the FROM clause</secondary>
2001-05-13 00:51:36 +02:00
</indexterm>
<indexterm>
2003-08-31 19:32:24 +02:00
<primary>label</primary>
<see>alias</see>
2001-05-13 00:51:36 +02:00
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
A temporary name can be given to tables and complex table
references to be used for references to the derived table in
2004-12-23 06:37:40 +01:00
the rest of the query. This is called a <firstterm>table
2001-01-23 00:34:33 +01:00
alias</firstterm>.
2002-09-20 20:39:41 +02:00
</para>
<para>
To create a table alias, write
2001-01-23 00:34:33 +01:00
<synopsis>
FROM <replaceable>table_reference</replaceable> AS <replaceable>alias</replaceable>
</synopsis>
2002-09-20 20:39:41 +02:00
or
<synopsis>
FROM <replaceable>table_reference</replaceable> <replaceable>alias</replaceable>
</synopsis>
2008-02-15 23:17:06 +01:00
The <literal>AS</literal> key word is optional noise.
2002-09-20 20:39:41 +02:00
<replaceable>alias</replaceable> can be any identifier.
</para>
<para>
A typical application of table aliases is to assign short
identifiers to long table names to keep the join clauses
readable. For example:
<programlisting>
SELECT * FROM some_very_long_table_name s JOIN another_fairly_long_name a ON s.id = a.num;
</programlisting>
</para>
<para>
2009-10-21 22:22:38 +02:00
The alias becomes the new name of the table reference so far as the
current query is concerned — it is not allowed to refer to the
table by the original name elsewhere in the query. Thus, this is not
valid:
2001-01-23 00:34:33 +01:00
<programlisting>
2009-10-21 22:22:38 +02:00
SELECT * FROM my_table AS m WHERE my_table.a > 5; -- wrong
2001-01-23 00:34:33 +01:00
</programlisting>
2002-09-20 20:39:41 +02:00
</para>
<para>
2001-01-23 00:34:33 +01:00
Table aliases are mainly for notational convenience, but it is
2007-02-01 01:28:19 +01:00
necessary to use them when joining a table to itself, e.g.:
2001-01-23 00:34:33 +01:00
<programlisting>
2006-09-18 21:54:01 +02:00
SELECT * FROM people AS mother JOIN people AS child ON mother.id = child.mother_id;
2001-01-23 00:34:33 +01:00
</programlisting>
Additionally, an alias is required if the table reference is a
2017-11-23 15:39:47 +01:00
subquery (see <xref linkend="queries-subqueries"/>).
2001-01-23 00:34:33 +01:00
</para>
<para>
2006-09-18 21:54:01 +02:00
Parentheses are used to resolve ambiguities. In the following example,
the first statement assigns the alias <literal>b</literal> to the second
2017-10-09 03:44:17 +02:00
instance of <literal>my_table</literal>, but the second statement assigns the
2006-09-18 21:54:01 +02:00
alias to the result of the join:
2001-01-23 00:34:33 +01:00
<programlisting>
2006-09-18 21:54:01 +02:00
SELECT * FROM my_table AS a CROSS JOIN my_table AS b ...
2001-01-23 00:34:33 +01:00
SELECT * FROM (my_table AS a CROSS JOIN my_table) AS b ...
</programlisting>
</para>
<para>
2004-12-23 06:37:40 +01:00
Another form of table aliasing gives temporary names to the columns of
the table, as well as the table itself:
2001-01-23 00:34:33 +01:00
<synopsis>
FROM <replaceable>table_reference</replaceable> <optional>AS</optional> <replaceable>alias</replaceable> ( <replaceable>column1</replaceable> <optional>, <replaceable>column2</replaceable> <optional>, ...</optional></optional> )
</synopsis>
2002-09-20 20:39:41 +02:00
If fewer column aliases are specified than the actual table has
columns, the remaining columns are not renamed. This syntax is
especially useful for self-joins or subqueries.
2001-01-23 00:34:33 +01:00
</para>
2001-02-15 05:10:54 +01:00
<para>
2017-10-09 03:44:17 +02:00
When an alias is applied to the output of a <literal>JOIN</literal>
2009-04-27 18:27:36 +02:00
clause, the alias hides the original
2017-10-09 03:44:17 +02:00
name(s) within the <literal>JOIN</literal>. For example:
2001-02-15 05:10:54 +01:00
<programlisting>
SELECT a.* FROM my_table AS a JOIN your_table AS b ON ...
</programlisting>
2007-02-01 01:28:19 +01:00
is valid SQL, but:
2001-02-15 05:10:54 +01:00
<programlisting>
SELECT a.* FROM (my_table AS a JOIN your_table AS b ON ...) AS c
</programlisting>
2017-10-09 03:44:17 +02:00
is not valid; the table alias <literal>a</literal> is not visible
outside the alias <literal>c</literal>.
2001-02-15 05:10:54 +01:00
</para>
2001-01-23 00:34:33 +01:00
</sect3>
2002-09-20 20:39:41 +02:00
<sect3 id="queries-subqueries">
<title>Subqueries</title>
<indexterm zone="queries-subqueries">
2003-08-31 19:32:24 +02:00
<primary>subquery</primary>
2002-09-20 20:39:41 +02:00
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
2002-09-20 20:39:41 +02:00
Subqueries specifying a derived table must be enclosed in
parentheses and <emphasis>must</emphasis> be assigned a table
2017-11-23 15:39:47 +01:00
alias name (as in <xref linkend="queries-table-aliases"/>). For
2002-09-20 20:39:41 +02:00
example:
2001-01-23 00:34:33 +01:00
<programlisting>
2002-09-20 20:39:41 +02:00
FROM (SELECT * FROM table1) AS alias_name
2001-01-23 00:34:33 +01:00
</programlisting>
2002-09-20 20:39:41 +02:00
</para>
2001-01-23 00:34:33 +01:00
2002-09-20 20:39:41 +02:00
<para>
This example is equivalent to <literal>FROM table1 AS
Wording cleanup for error messages. Also change can't -> cannot.
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
2007-02-01 20:10:30 +01:00
alias_name</literal>. More interesting cases, which cannot be
2002-09-20 20:39:41 +02:00
reduced to a plain join, arise when the subquery involves
grouping or aggregation.
2001-01-23 00:34:33 +01:00
</para>
2006-09-18 21:54:01 +02:00
<para>
2017-10-09 03:44:17 +02:00
A subquery can also be a <command>VALUES</command> list:
2006-09-18 21:54:01 +02:00
<programlisting>
FROM (VALUES ('anne', 'smith'), ('bob', 'jones'), ('joe', 'blow'))
AS names(first, last)
</programlisting>
Again, a table alias is required. Assigning alias names to the columns
2017-10-09 03:44:17 +02:00
of the <command>VALUES</command> list is optional, but is good practice.
2017-11-23 15:39:47 +01:00
For more information see <xref linkend="queries-values"/>.
2006-09-18 21:54:01 +02:00
</para>
2001-01-23 00:34:33 +01:00
</sect3>
2003-04-10 03:22:45 +02:00
<sect3 id="queries-tablefunctions">
<title>Table Functions</title>
2017-10-09 03:44:17 +02:00
<indexterm zone="queries-tablefunctions"><primary>table function</primary></indexterm>
2003-04-10 03:22:45 +02:00
2003-08-31 19:32:24 +02:00
<indexterm zone="queries-tablefunctions">
2017-10-09 03:44:17 +02:00
<primary>function</primary>
<secondary>in the FROM clause</secondary>
2003-08-31 19:32:24 +02:00
</indexterm>
2003-04-10 03:22:45 +02:00
<para>
Table functions are functions that produce a set of rows, made up
of either base data types (scalar types) or composite data types
(table rows). They are used like a table, view, or subquery in
2017-10-09 03:44:17 +02:00
the <literal>FROM</literal> clause of a query. Columns returned by table
functions can be included in <literal>SELECT</literal>,
<literal>JOIN</literal>, or <literal>WHERE</literal> clauses in the same manner
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
as columns of a table, view, or subquery.
2003-04-10 03:22:45 +02:00
</para>
<para>
2017-10-09 03:44:17 +02:00
Table functions may also be combined using the <literal>ROWS FROM</literal>
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
syntax, with the results returned in parallel columns; the number of
result rows in this case is that of the largest function result, with
2014-07-17 04:20:15 +02:00
smaller results padded with null values to match.
2003-04-10 03:22:45 +02:00
</para>
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
<synopsis>
<replaceable>function_call</replaceable> <optional>WITH ORDINALITY</optional> <optional><optional>AS</optional> <replaceable>table_alias</replaceable> <optional>(<replaceable>column_alias</replaceable> <optional>, ... </optional>)</optional></optional>
2013-12-10 15:34:37 +01:00
ROWS FROM( <replaceable>function_call</replaceable> <optional>, ... </optional> ) <optional>WITH ORDINALITY</optional> <optional><optional>AS</optional> <replaceable>table_alias</replaceable> <optional>(<replaceable>column_alias</replaceable> <optional>, ... </optional>)</optional></optional>
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
</synopsis>
<para>
If the <literal>WITH ORDINALITY</literal> clause is specified, an
additional column of type <type>bigint</type> will be added to the
function result columns. This column numbers the rows of the function
result set, starting from 1. (This is a generalization of the
SQL-standard syntax for <literal>UNNEST ... WITH ORDINALITY</literal>.)
2017-10-09 03:44:17 +02:00
By default, the ordinal column is called <literal>ordinality</literal>, but
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
a different column name can be assigned to it using
an <literal>AS</literal> clause.
</para>
<para>
The special table function <literal>UNNEST</literal> may be called with
any number of array parameters, and it returns a corresponding number of
columns, as if <literal>UNNEST</literal>
2017-11-23 15:39:47 +01:00
(<xref linkend="functions-array"/>) had been called on each parameter
2013-12-10 15:34:37 +01:00
separately and combined using the <literal>ROWS FROM</literal> construct.
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
</para>
<synopsis>
UNNEST( <replaceable>array_expression</replaceable> <optional>, ... </optional> ) <optional>WITH ORDINALITY</optional> <optional><optional>AS</optional> <replaceable>table_alias</replaceable> <optional>(<replaceable>column_alias</replaceable> <optional>, ... </optional>)</optional></optional>
</synopsis>
2003-04-10 03:22:45 +02:00
<para>
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
If no <replaceable>table_alias</replaceable> is specified, the function
2017-10-09 03:44:17 +02:00
name is used as the table name; in the case of a <literal>ROWS FROM()</literal>
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
construct, the first function's name is used.
</para>
<para>
If column aliases are not supplied, then for a function returning a base
data type, the column name is also the same as the function name. For a
function returning a composite type, the result columns get the names
of the individual attributes of the type.
2003-04-10 03:22:45 +02:00
</para>
<para>
Some examples:
<programlisting>
CREATE TABLE foo (fooid int, foosubid int, fooname text);
2004-05-17 01:22:08 +02:00
CREATE FUNCTION getfoo(int) RETURNS SETOF foo AS $$
2003-04-10 03:22:45 +02:00
SELECT * FROM foo WHERE fooid = $1;
2004-05-17 01:22:08 +02:00
$$ LANGUAGE SQL;
2003-04-10 03:22:45 +02:00
SELECT * FROM getfoo(1) AS t1;
SELECT * FROM foo
2009-04-27 18:27:36 +02:00
WHERE foosubid IN (
SELECT foosubid
FROM getfoo(foo.fooid) z
WHERE z.fooid = foo.fooid
);
2003-04-10 03:22:45 +02:00
CREATE VIEW vw_getfoo AS SELECT * FROM getfoo(1);
2004-12-23 06:37:40 +01:00
2003-04-10 03:22:45 +02:00
SELECT * FROM vw_getfoo;
</programlisting>
</para>
<para>
In some cases it is useful to define table functions that can
return different column sets depending on how they are invoked.
To support this, the table function can be declared as returning
2020-10-05 22:27:33 +02:00
the pseudo-type <type>record</type> with no <literal>OUT</literal>
parameters. When such a function is used in
2003-04-10 03:22:45 +02:00
a query, the expected row structure must be specified in the
query itself, so that the system can know how to parse and plan
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
the query. This syntax looks like:
</para>
<synopsis>
<replaceable>function_call</replaceable> <optional>AS</optional> <replaceable>alias</replaceable> (<replaceable>column_definition</replaceable> <optional>, ... </optional>)
<replaceable>function_call</replaceable> AS <optional><replaceable>alias</replaceable></optional> (<replaceable>column_definition</replaceable> <optional>, ... </optional>)
2013-12-10 15:34:37 +01:00
ROWS FROM( ... <replaceable>function_call</replaceable> AS (<replaceable>column_definition</replaceable> <optional>, ... </optional>) <optional>, ... </optional> )
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
</synopsis>
<para>
2017-10-09 03:44:17 +02:00
When not using the <literal>ROWS FROM()</literal> syntax,
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
the <replaceable>column_definition</replaceable> list replaces the column
2017-10-09 03:44:17 +02:00
alias list that could otherwise be attached to the <literal>FROM</literal>
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
item; the names in the column definitions serve as column aliases.
2017-10-09 03:44:17 +02:00
When using the <literal>ROWS FROM()</literal> syntax,
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
a <replaceable>column_definition</replaceable> list can be attached to
each member function separately; or if there is only one member function
2017-10-09 03:44:17 +02:00
and no <literal>WITH ORDINALITY</literal> clause,
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
a <replaceable>column_definition</replaceable> list can be written in
2017-10-09 03:44:17 +02:00
place of a column alias list following <literal>ROWS FROM()</literal>.
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
</para>
<para>
Consider this example:
2003-04-10 03:22:45 +02:00
<programlisting>
SELECT *
2009-04-27 18:27:36 +02:00
FROM dblink('dbname=mydb', 'SELECT proname, prosrc FROM pg_proc')
2003-04-10 03:22:45 +02:00
AS t1(proname name, prosrc text)
WHERE proname LIKE 'bytea%';
</programlisting>
2017-11-23 15:39:47 +01:00
The <xref linkend="contrib-dblink-function"/> function
(part of the <xref linkend="dblink"/> module) executes
2011-01-26 15:22:21 +01:00
a remote query. It is declared to return
2017-10-09 03:44:17 +02:00
<type>record</type> since it might be used for any kind of query.
2003-04-10 03:22:45 +02:00
The actual column set must be specified in the calling query so
2017-10-09 03:44:17 +02:00
that the parser knows, for example, what <literal>*</literal> should
2003-04-10 03:22:45 +02:00
expand to.
</para>
2020-10-05 22:27:33 +02:00
<para>
This example uses <literal>ROWS FROM</literal>:
<programlisting>
SELECT *
FROM ROWS FROM
(
json_to_recordset('[{"a":40,"b":"foo"},{"a":"100","b":"bar"}]')
AS (a INTEGER, b TEXT),
generate_series(1, 3)
) AS x (p, q, s)
ORDER BY p;
p | q | s
-----+-----+---
40 | foo | 1
100 | bar | 2
| | 3
</programlisting>
It joins two functions into a single <literal>FROM</literal>
target. <function>json_to_recordset()</function> is instructed
to return two columns, the first <type>integer</type>
and the second <type>text</type>. The result of
<function>generate_series()</function> is used directly.
The <literal>ORDER BY</literal> clause sorts the column values
as integers.
</para>
2003-04-10 03:22:45 +02:00
</sect3>
2012-08-08 01:02:54 +02:00
<sect3 id="queries-lateral">
2017-10-09 03:44:17 +02:00
<title><literal>LATERAL</literal> Subqueries</title>
2012-08-08 01:02:54 +02:00
<indexterm zone="queries-lateral">
2017-10-09 03:44:17 +02:00
<primary>LATERAL</primary>
<secondary>in the FROM clause</secondary>
2012-08-08 01:02:54 +02:00
</indexterm>
<para>
2017-10-09 03:44:17 +02:00
Subqueries appearing in <literal>FROM</literal> can be
preceded by the key word <literal>LATERAL</literal>. This allows them to
reference columns provided by preceding <literal>FROM</literal> items.
2013-01-26 22:18:42 +01:00
(Without <literal>LATERAL</literal>, each subquery is
2012-08-08 01:02:54 +02:00
evaluated independently and so cannot cross-reference any other
2017-10-09 03:44:17 +02:00
<literal>FROM</literal> item.)
2013-01-26 22:18:42 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
Table functions appearing in <literal>FROM</literal> can also be
preceded by the key word <literal>LATERAL</literal>, but for functions the
2013-01-26 22:18:42 +01:00
key word is optional; the function's arguments can contain references
2017-10-09 03:44:17 +02:00
to columns provided by preceding <literal>FROM</literal> items in any case.
2013-01-26 22:18:42 +01:00
</para>
<para>
2012-08-08 01:02:54 +02:00
A <literal>LATERAL</literal> item can appear at top level in the
2017-10-09 03:44:17 +02:00
<literal>FROM</literal> list, or within a <literal>JOIN</literal> tree. In the latter
2012-08-08 01:02:54 +02:00
case it can also refer to any items that are on the left-hand side of a
2017-10-09 03:44:17 +02:00
<literal>JOIN</literal> that it is on the right-hand side of.
2012-08-08 01:02:54 +02:00
</para>
<para>
2017-10-09 03:44:17 +02:00
When a <literal>FROM</literal> item contains <literal>LATERAL</literal>
2012-08-08 01:02:54 +02:00
cross-references, evaluation proceeds as follows: for each row of the
2017-10-09 03:44:17 +02:00
<literal>FROM</literal> item providing the cross-referenced column(s), or
set of rows of multiple <literal>FROM</literal> items providing the
2012-08-08 01:02:54 +02:00
columns, the <literal>LATERAL</literal> item is evaluated using that
row or row set's values of the columns. The resulting row(s) are
joined as usual with the rows they were computed from. This is
repeated for each row or set of rows from the column source table(s).
</para>
<para>
A trivial example of <literal>LATERAL</literal> is
<programlisting>
SELECT * FROM foo, LATERAL (SELECT * FROM bar WHERE bar.id = foo.bar_id) ss;
</programlisting>
This is not especially useful since it has exactly the same result as
the more conventional
<programlisting>
SELECT * FROM foo, bar WHERE bar.id = foo.bar_id;
</programlisting>
<literal>LATERAL</literal> is primarily useful when the cross-referenced
column is necessary for computing the row(s) to be joined. A common
application is providing an argument value for a set-returning function.
2017-10-09 03:44:17 +02:00
For example, supposing that <function>vertices(polygon)</function> returns the
2012-08-08 01:02:54 +02:00
set of vertices of a polygon, we could identify close-together vertices
of polygons stored in a table with:
<programlisting>
SELECT p1.id, p2.id, v1, v2
FROM polygons p1, polygons p2,
LATERAL vertices(p1.poly) v1,
LATERAL vertices(p2.poly) v2
WHERE (v1 <-> v2) < 10 AND p1.id != p2.id;
</programlisting>
This query could also be written
<programlisting>
SELECT p1.id, p2.id, v1, v2
FROM polygons p1 CROSS JOIN LATERAL vertices(p1.poly) v1,
polygons p2 CROSS JOIN LATERAL vertices(p2.poly) v2
WHERE (v1 <-> v2) < 10 AND p1.id != p2.id;
</programlisting>
2013-01-26 22:18:42 +01:00
or in several other equivalent formulations. (As already mentioned,
2017-10-09 03:44:17 +02:00
the <literal>LATERAL</literal> key word is unnecessary in this example, but
2013-01-26 22:18:42 +01:00
we use it for clarity.)
2012-08-08 01:02:54 +02:00
</para>
<para>
2017-10-09 03:44:17 +02:00
It is often particularly handy to <literal>LEFT JOIN</literal> to a
2012-08-08 01:02:54 +02:00
<literal>LATERAL</literal> subquery, so that source rows will appear in
the result even if the <literal>LATERAL</literal> subquery produces no
2017-10-09 03:44:17 +02:00
rows for them. For example, if <function>get_product_names()</function> returns
2012-08-08 01:02:54 +02:00
the names of products made by a manufacturer, but some manufacturers in
our table currently produce no products, we could find out which ones
those are like this:
<programlisting>
SELECT m.name
FROM manufacturers m LEFT JOIN LATERAL get_product_names(m.id) pname ON true
WHERE pname IS NULL;
</programlisting>
</para>
</sect3>
2001-01-23 00:34:33 +01:00
</sect2>
<sect2 id="queries-where">
2003-11-04 10:55:39 +01:00
<title>The <literal>WHERE</literal> Clause</title>
2001-01-23 00:34:33 +01:00
2001-05-13 00:51:36 +02:00
<indexterm zone="queries-where">
2003-08-31 19:32:24 +02:00
<primary>WHERE</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
Doc: fix "Unresolved ID reference" warnings, clean up man page cross-refs.
Use xreflabel attributes instead of endterm attributes to control the
appearance of links to subsections of SQL command reference pages.
This is simpler, it matches what we do elsewhere (e.g. for GUC variables),
and it doesn't draw "Unresolved ID reference" warnings from the PDF
toolchain.
Fix some places where the text was absolutely dependent on an <xref>
rendering exactly so, by using a <link> around the required text
instead. At least one of those spots had already been turned into
bad grammar by subsequent changes, and the whole idea is just too
fragile for my taste. <xref> does NOT have fixed output, don't write
as if it does.
Consistently include a page-level link in cross-man-page references,
because otherwise they are useless/nonsensical in man-page output.
Likewise, be consistent about mentioning "below" or "above" in same-page
references; we were doing that in about 90% of the cases, but now it's
100%.
Also get rid of another nonfunctional-in-PDF idea, of making
cross-references to functions by sticking ID tags on <row> constructs.
We can put the IDs on <indexterm>s instead --- which is probably not any
more sensible in abstract terms, but it works where the other doesn't.
(There is talk of attaching cross-reference IDs to most or all of
the docs' function descriptions, but for now I just fixed the two
that exist.)
Discussion: https://postgr.es/m/14480.1589154358@sss.pgh.pa.us
2020-05-11 20:15:49 +02:00
The syntax of the <link linkend="sql-where"><literal>WHERE</literal></link>
clause is
2001-01-23 00:34:33 +01:00
<synopsis>
2001-03-25 00:03:26 +01:00
WHERE <replaceable>search_condition</replaceable>
2001-01-23 00:34:33 +01:00
</synopsis>
2001-03-25 00:03:26 +01:00
where <replaceable>search_condition</replaceable> is any value
2017-11-23 15:39:47 +01:00
expression (see <xref linkend="sql-expressions"/>) that
2001-01-23 00:34:33 +01:00
returns a value of type <type>boolean</type>.
</para>
<para>
2017-10-09 03:44:17 +02:00
After the processing of the <literal>FROM</literal> clause is done, each
2002-09-20 20:39:41 +02:00
row of the derived virtual table is checked against the search
condition. If the result of the condition is true, the row is
2009-04-27 18:27:36 +02:00
kept in the output table, otherwise (i.e., if the result is
2002-09-20 20:39:41 +02:00
false or null) it is discarded. The search condition typically
2009-04-27 18:27:36 +02:00
references at least one column of the table generated in the
2017-10-09 03:44:17 +02:00
<literal>FROM</literal> clause; this is not required, but otherwise the
<literal>WHERE</literal> clause will be fairly useless.
2001-01-23 00:34:33 +01:00
</para>
<note>
<para>
2004-12-23 06:37:40 +01:00
The join condition of an inner join can be written either in
2017-10-09 03:44:17 +02:00
the <literal>WHERE</literal> clause or in the <literal>JOIN</literal> clause.
2004-12-23 06:37:40 +01:00
For example, these table expressions are equivalent:
2001-01-23 00:34:33 +01:00
<programlisting>
FROM a, b WHERE a.id = b.id AND b.val > 5
</programlisting>
2007-02-01 01:28:19 +01:00
and:
2001-01-23 00:34:33 +01:00
<programlisting>
FROM a INNER JOIN b ON (a.id = b.id) WHERE b.val > 5
</programlisting>
2007-02-01 01:28:19 +01:00
or perhaps even:
2001-01-23 00:34:33 +01:00
<programlisting>
FROM a NATURAL JOIN b WHERE b.val > 5
</programlisting>
2002-09-20 20:39:41 +02:00
Which one of these you use is mainly a matter of style. The
2017-10-09 03:44:17 +02:00
<literal>JOIN</literal> syntax in the <literal>FROM</literal> clause is
2009-04-27 18:27:36 +02:00
probably not as portable to other SQL database management systems,
even though it is in the SQL standard. For
outer joins there is no choice: they must be done in
2017-10-09 03:44:17 +02:00
the <literal>FROM</literal> clause. The <literal>ON</literal> or <literal>USING</literal>
clause of an outer join is <emphasis>not</emphasis> equivalent to a
<literal>WHERE</literal> condition, because it results in the addition
2002-09-20 20:39:41 +02:00
of rows (for unmatched input rows) as well as the removal of rows
2009-06-17 23:58:49 +02:00
in the final result.
2001-01-23 00:34:33 +01:00
</para>
</note>
2002-09-20 20:39:41 +02:00
<para>
Here are some examples of <literal>WHERE</literal> clauses:
2001-01-23 00:34:33 +01:00
<programlisting>
2005-01-22 23:56:36 +01:00
SELECT ... FROM fdt WHERE c1 > 5
2001-01-23 00:34:33 +01:00
2002-09-20 20:39:41 +02:00
SELECT ... FROM fdt WHERE c1 IN (1, 2, 3)
2001-01-23 00:34:33 +01:00
2002-09-20 20:39:41 +02:00
SELECT ... FROM fdt WHERE c1 IN (SELECT c1 FROM t2)
2001-01-23 00:34:33 +01:00
2002-09-20 20:39:41 +02:00
SELECT ... FROM fdt WHERE c1 IN (SELECT c3 FROM t2 WHERE c2 = fdt.c1 + 10)
2001-01-23 00:34:33 +01:00
2002-09-20 20:39:41 +02:00
SELECT ... FROM fdt WHERE c1 BETWEEN (SELECT c3 FROM t2 WHERE c2 = fdt.c1 + 10) AND 100
2005-01-22 23:56:36 +01:00
SELECT ... FROM fdt WHERE EXISTS (SELECT c1 FROM t2 WHERE c2 > fdt.c1)
2002-09-20 20:39:41 +02:00
</programlisting>
2009-06-17 23:58:49 +02:00
<literal>fdt</literal> is the table derived in the
2017-10-09 03:44:17 +02:00
<literal>FROM</literal> clause. Rows that do not meet the search
condition of the <literal>WHERE</literal> clause are eliminated from
2002-09-20 20:39:41 +02:00
<literal>fdt</literal>. Notice the use of scalar subqueries as
2001-09-09 19:21:59 +02:00
value expressions. Just like any other query, the subqueries can
2003-03-13 02:30:29 +01:00
employ complex table expressions. Notice also how
2002-09-20 20:39:41 +02:00
<literal>fdt</literal> is referenced in the subqueries.
2017-10-09 03:44:17 +02:00
Qualifying <literal>c1</literal> as <literal>fdt.c1</literal> is only necessary
if <literal>c1</literal> is also the name of a column in the derived
2003-03-13 02:30:29 +01:00
input table of the subquery. But qualifying the column name adds
clarity even when it is not needed. This example shows how the column
2001-09-09 19:21:59 +02:00
naming scope of an outer query extends into its inner queries.
2001-01-23 00:34:33 +01:00
</para>
</sect2>
<sect2 id="queries-group">
2003-11-04 10:55:39 +01:00
<title>The <literal>GROUP BY</literal> and <literal>HAVING</literal> Clauses</title>
2001-01-23 00:34:33 +01:00
2001-05-13 00:51:36 +02:00
<indexterm zone="queries-group">
2003-08-31 19:32:24 +02:00
<primary>GROUP BY</primary>
</indexterm>
<indexterm zone="queries-group">
<primary>grouping</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
2017-10-09 03:44:17 +02:00
After passing the <literal>WHERE</literal> filter, the derived input
table might be subject to grouping, using the <literal>GROUP BY</literal>
clause, and elimination of group rows using the <literal>HAVING</literal>
2002-09-20 20:39:41 +02:00
clause.
2001-01-23 00:34:33 +01:00
</para>
<synopsis>
2001-10-09 20:46:00 +02:00
SELECT <replaceable>select_list</replaceable>
FROM ...
<optional>WHERE ...</optional>
GROUP BY <replaceable>grouping_column_reference</replaceable> <optional>, <replaceable>grouping_column_reference</replaceable></optional>...
2001-01-23 00:34:33 +01:00
</synopsis>
<para>
Doc: fix "Unresolved ID reference" warnings, clean up man page cross-refs.
Use xreflabel attributes instead of endterm attributes to control the
appearance of links to subsections of SQL command reference pages.
This is simpler, it matches what we do elsewhere (e.g. for GUC variables),
and it doesn't draw "Unresolved ID reference" warnings from the PDF
toolchain.
Fix some places where the text was absolutely dependent on an <xref>
rendering exactly so, by using a <link> around the required text
instead. At least one of those spots had already been turned into
bad grammar by subsequent changes, and the whole idea is just too
fragile for my taste. <xref> does NOT have fixed output, don't write
as if it does.
Consistently include a page-level link in cross-man-page references,
because otherwise they are useless/nonsensical in man-page output.
Likewise, be consistent about mentioning "below" or "above" in same-page
references; we were doing that in about 90% of the cases, but now it's
100%.
Also get rid of another nonfunctional-in-PDF idea, of making
cross-references to functions by sticking ID tags on <row> constructs.
We can put the IDs on <indexterm>s instead --- which is probably not any
more sensible in abstract terms, but it works where the other doesn't.
(There is talk of attaching cross-reference IDs to most or all of
the docs' function descriptions, but for now I just fixed the two
that exist.)
Discussion: https://postgr.es/m/14480.1589154358@sss.pgh.pa.us
2020-05-11 20:15:49 +02:00
The <link linkend="sql-groupby"><literal>GROUP BY</literal></link> clause is
2009-04-27 18:27:36 +02:00
used to group together those rows in a table that have the same
2004-03-03 23:22:24 +01:00
values in all the columns listed. The order in which the columns
2004-12-23 06:37:40 +01:00
are listed does not matter. The effect is to combine each set
2009-04-27 18:27:36 +02:00
of rows having common values into one group row that
represents all rows in the group. This is done to
2004-03-03 23:22:24 +01:00
eliminate redundancy in the output and/or compute aggregates that
apply to these groups. For instance:
2002-09-20 20:39:41 +02:00
<screen>
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM test1;</userinput>
2002-09-20 20:39:41 +02:00
x | y
---+---
a | 3
c | 2
b | 5
a | 1
(4 rows)
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT x FROM test1 GROUP BY x;</userinput>
2002-09-20 20:39:41 +02:00
x
---
a
b
c
(3 rows)
</screen>
2001-01-23 00:34:33 +01:00
</para>
2006-09-18 21:54:01 +02:00
2001-01-23 00:34:33 +01:00
<para>
2002-09-20 20:39:41 +02:00
In the second query, we could not have written <literal>SELECT *
2002-10-20 07:05:46 +02:00
FROM test1 GROUP BY x</literal>, because there is no single value
2017-10-09 03:44:17 +02:00
for the column <literal>y</literal> that could be associated with each
2002-11-11 21:14:04 +01:00
group. The grouped-by columns can be referenced in the select list since
2004-12-23 06:37:40 +01:00
they have a single value in each group.
2002-11-11 21:14:04 +01:00
</para>
<para>
In general, if a table is grouped, columns that are not
2017-10-09 03:44:17 +02:00
listed in <literal>GROUP BY</literal> cannot be referenced except in aggregate
2002-10-20 07:05:46 +02:00
expressions. An example with aggregate expressions is:
2002-09-20 20:39:41 +02:00
<screen>
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT x, sum(y) FROM test1 GROUP BY x;</userinput>
2002-09-20 20:39:41 +02:00
x | sum
---+-----
a | 4
b | 5
c | 2
(3 rows)
</screen>
2003-03-13 02:30:29 +01:00
Here <literal>sum</literal> is an aggregate function that
2002-09-20 20:39:41 +02:00
computes a single value over the entire group. More information
about the available aggregate functions can be found in <xref
2017-11-23 15:39:47 +01:00
linkend="functions-aggregate"/>.
2001-01-23 00:34:33 +01:00
</para>
2002-10-20 07:05:46 +02:00
<tip>
2002-09-20 20:39:41 +02:00
<para>
Grouping without aggregate expressions effectively calculates the
2009-06-17 23:58:49 +02:00
set of distinct values in a column. This can also be achieved
2017-10-09 03:44:17 +02:00
using the <literal>DISTINCT</literal> clause (see <xref
2017-11-23 15:39:47 +01:00
linkend="queries-distinct"/>).
2002-09-20 20:39:41 +02:00
</para>
2002-10-20 07:05:46 +02:00
</tip>
2002-09-20 20:39:41 +02:00
<para>
2003-03-13 02:30:29 +01:00
Here is another example: it calculates the total sales for each
2009-04-27 18:27:36 +02:00
product (rather than the total sales of all products):
2001-01-23 00:34:33 +01:00
<programlisting>
2002-09-20 20:39:41 +02:00
SELECT product_id, p.name, (sum(s.units) * p.price) AS sales
FROM products p LEFT JOIN sales s USING (product_id)
2002-10-20 07:05:46 +02:00
GROUP BY product_id, p.name, p.price;
2001-01-23 00:34:33 +01:00
</programlisting>
2002-10-20 07:05:46 +02:00
In this example, the columns <literal>product_id</literal>,
2002-09-20 20:39:41 +02:00
<literal>p.name</literal>, and <literal>p.price</literal> must be
2017-10-09 03:44:17 +02:00
in the <literal>GROUP BY</literal> clause since they are referenced in
2010-08-07 04:44:09 +02:00
the query select list (but see below). The column
2017-10-09 03:44:17 +02:00
<literal>s.units</literal> does not have to be in the <literal>GROUP
BY</literal> list since it is only used in an aggregate expression
2003-03-13 02:30:29 +01:00
(<literal>sum(...)</literal>), which represents the sales
of a product. For each product, the query returns a summary row about
2001-01-23 00:34:33 +01:00
all sales of the product.
</para>
2011-04-08 17:36:05 +02:00
<indexterm><primary>functional dependency</primary></indexterm>
2010-08-07 04:44:09 +02:00
<para>
2011-04-08 17:36:05 +02:00
If the products table is set up so that, say,
<literal>product_id</literal> is the primary key, then it would be
enough to group by <literal>product_id</literal> in the above example,
since name and price would be <firstterm>functionally
dependent</firstterm> on the product ID, and so there would be no
ambiguity about which name and price value to return for each product
ID group.
2010-08-07 04:44:09 +02:00
</para>
2001-01-23 00:34:33 +01:00
<para>
2017-10-09 03:44:17 +02:00
In strict SQL, <literal>GROUP BY</literal> can only group by columns of
2002-09-20 20:39:41 +02:00
the source table but <productname>PostgreSQL</productname> extends
2017-10-09 03:44:17 +02:00
this to also allow <literal>GROUP BY</literal> to group by columns in the
2002-09-20 20:39:41 +02:00
select list. Grouping by value expressions instead of simple
column names is also allowed.
2001-01-23 00:34:33 +01:00
</para>
2003-08-31 19:32:24 +02:00
<indexterm>
<primary>HAVING</primary>
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
2009-04-27 18:27:36 +02:00
If a table has been grouped using <literal>GROUP BY</literal>,
but only certain groups are of interest, the
2002-09-20 20:39:41 +02:00
<literal>HAVING</literal> clause can be used, much like a
2017-10-09 03:44:17 +02:00
<literal>WHERE</literal> clause, to eliminate groups from the result.
2009-04-27 18:27:36 +02:00
The syntax is:
2001-01-23 00:34:33 +01:00
<synopsis>
SELECT <replaceable>select_list</replaceable> FROM ... <optional>WHERE ...</optional> GROUP BY ... HAVING <replaceable>boolean_expression</replaceable>
</synopsis>
2017-10-09 03:44:17 +02:00
Expressions in the <literal>HAVING</literal> clause can refer both to
2002-10-20 07:05:46 +02:00
grouped expressions and to ungrouped expressions (which necessarily
2002-09-20 20:39:41 +02:00
involve an aggregate function).
2001-01-23 00:34:33 +01:00
</para>
<para>
Example:
2002-09-20 20:39:41 +02:00
<screen>
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT x, sum(y) FROM test1 GROUP BY x HAVING sum(y) > 3;</userinput>
2002-09-20 20:39:41 +02:00
x | sum
---+-----
a | 4
b | 5
(2 rows)
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT x, sum(y) FROM test1 GROUP BY x HAVING x < 'c';</userinput>
2002-09-20 20:39:41 +02:00
x | sum
---+-----
a | 4
b | 5
(2 rows)
</screen>
</para>
<para>
Again, a more realistic example:
2001-01-23 00:34:33 +01:00
<programlisting>
2002-09-20 20:39:41 +02:00
SELECT product_id, p.name, (sum(s.units) * (p.price - p.cost)) AS profit
2002-10-20 07:05:46 +02:00
FROM products p LEFT JOIN sales s USING (product_id)
2005-01-22 23:56:36 +01:00
WHERE s.date > CURRENT_DATE - INTERVAL '4 weeks'
2002-09-20 20:39:41 +02:00
GROUP BY product_id, p.name, p.price, p.cost
2005-01-22 23:56:36 +01:00
HAVING sum(p.price * s.units) > 5000;
2001-01-23 00:34:33 +01:00
</programlisting>
2017-10-09 03:44:17 +02:00
In the example above, the <literal>WHERE</literal> clause is selecting
2003-03-13 02:30:29 +01:00
rows by a column that is not grouped (the expression is only true for
2017-10-09 03:44:17 +02:00
sales during the last four weeks), while the <literal>HAVING</literal>
2002-09-20 20:39:41 +02:00
clause restricts the output to groups with total gross sales over
5000. Note that the aggregate expressions do not necessarily need
2003-03-13 02:30:29 +01:00
to be the same in all parts of the query.
2001-01-23 00:34:33 +01:00
</para>
2008-12-28 19:54:01 +01:00
<para>
2017-10-09 03:44:17 +02:00
If a query contains aggregate function calls, but no <literal>GROUP BY</literal>
2008-12-28 19:54:01 +01:00
clause, grouping still occurs: the result is a single group row (or
perhaps no rows at all, if the single row is then eliminated by
2017-10-09 03:44:17 +02:00
<literal>HAVING</literal>).
The same is true if it contains a <literal>HAVING</literal> clause, even
without any aggregate function calls or <literal>GROUP BY</literal> clause.
2008-12-28 19:54:01 +01:00
</para>
</sect2>
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
<sect2 id="queries-grouping-sets">
2017-10-09 03:44:17 +02:00
<title><literal>GROUPING SETS</literal>, <literal>CUBE</literal>, and <literal>ROLLUP</literal></title>
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
<indexterm zone="queries-grouping-sets">
<primary>GROUPING SETS</primary>
</indexterm>
<indexterm zone="queries-grouping-sets">
<primary>CUBE</primary>
</indexterm>
<indexterm zone="queries-grouping-sets">
<primary>ROLLUP</primary>
</indexterm>
<para>
More complex grouping operations than those described above are possible
2017-10-09 03:44:17 +02:00
using the concept of <firstterm>grouping sets</firstterm>. The data selected by
the <literal>FROM</literal> and <literal>WHERE</literal> clauses is grouped separately
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
by each specified grouping set, aggregates computed for each group just as
2017-10-09 03:44:17 +02:00
for simple <literal>GROUP BY</literal> clauses, and then the results returned.
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
For example:
<screen>
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT * FROM items_sold;</userinput>
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
brand | size | sales
-------+------+-------
Foo | L | 10
Foo | M | 20
Bar | M | 15
Bar | L | 5
(4 rows)
2017-10-09 03:44:17 +02:00
<prompt>=></prompt> <userinput>SELECT brand, size, sum(sales) FROM items_sold GROUP BY GROUPING SETS ((brand), (size), ());</userinput>
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
brand | size | sum
-------+------+-----
Foo | | 30
Bar | | 20
| L | 15
| M | 35
| | 50
(5 rows)
</screen>
</para>
<para>
2017-10-09 03:44:17 +02:00
Each sublist of <literal>GROUPING SETS</literal> may specify zero or more columns
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
or expressions and is interpreted the same way as though it were directly
2017-10-09 03:44:17 +02:00
in the <literal>GROUP BY</literal> clause. An empty grouping set means that all
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
rows are aggregated down to a single group (which is output even if no
input rows were present), as described above for the case of aggregate
2017-10-09 03:44:17 +02:00
functions with no <literal>GROUP BY</literal> clause.
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
</para>
<para>
References to the grouping columns or expressions are replaced
2015-09-11 03:22:21 +02:00
by null values in result rows for grouping sets in which those
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
columns do not appear. To distinguish which grouping a particular output
2017-11-23 15:39:47 +01:00
row resulted from, see <xref linkend="functions-grouping-table"/>.
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
</para>
<para>
A shorthand notation is provided for specifying two common types of grouping set.
A clause of the form
<programlisting>
2017-10-09 03:44:17 +02:00
ROLLUP ( <replaceable>e1</replaceable>, <replaceable>e2</replaceable>, <replaceable>e3</replaceable>, ... )
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
</programlisting>
represents the given list of expressions and all prefixes of the list including
the empty list; thus it is equivalent to
<programlisting>
GROUPING SETS (
2017-10-09 03:44:17 +02:00
( <replaceable>e1</replaceable>, <replaceable>e2</replaceable>, <replaceable>e3</replaceable>, ... ),
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
...
2017-10-09 03:44:17 +02:00
( <replaceable>e1</replaceable>, <replaceable>e2</replaceable> ),
( <replaceable>e1</replaceable> ),
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
( )
)
</programlisting>
2020-09-01 00:33:37 +02:00
This is commonly used for analysis over hierarchical data; e.g., total
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
salary by department, division, and company-wide total.
</para>
<para>
A clause of the form
<programlisting>
2017-10-09 03:44:17 +02:00
CUBE ( <replaceable>e1</replaceable>, <replaceable>e2</replaceable>, ... )
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
</programlisting>
2020-09-01 00:33:37 +02:00
represents the given list and all of its possible subsets (i.e., the power
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
set). Thus
<programlisting>
CUBE ( a, b, c )
</programlisting>
is equivalent to
<programlisting>
GROUPING SETS (
( a, b, c ),
( a, b ),
( a, c ),
( a ),
( b, c ),
( b ),
( c ),
2015-11-17 00:59:55 +01:00
( )
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
)
</programlisting>
</para>
<para>
2017-10-09 03:44:17 +02:00
The individual elements of a <literal>CUBE</literal> or <literal>ROLLUP</literal>
2015-09-11 03:22:21 +02:00
clause may be either individual expressions, or sublists of elements in
parentheses. In the latter case, the sublists are treated as single
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
units for the purposes of generating the individual grouping sets.
For example:
<programlisting>
2015-11-17 00:59:55 +01:00
CUBE ( (a, b), (c, d) )
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
</programlisting>
is equivalent to
<programlisting>
GROUPING SETS (
2015-11-17 00:59:55 +01:00
( a, b, c, d ),
( a, b ),
( c, d ),
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
( )
)
</programlisting>
and
<programlisting>
2015-11-17 00:59:55 +01:00
ROLLUP ( a, (b, c), d )
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
</programlisting>
is equivalent to
<programlisting>
GROUPING SETS (
2015-11-17 00:59:55 +01:00
( a, b, c, d ),
( a, b, c ),
( a ),
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
( )
)
</programlisting>
</para>
<para>
2017-10-09 03:44:17 +02:00
The <literal>CUBE</literal> and <literal>ROLLUP</literal> constructs can be used either
directly in the <literal>GROUP BY</literal> clause, or nested inside a
<literal>GROUPING SETS</literal> clause. If one <literal>GROUPING SETS</literal> clause
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
is nested inside another, the effect is the same as if all the elements of
the inner clause had been written directly in the outer clause.
</para>
<para>
2017-10-09 03:44:17 +02:00
If multiple grouping items are specified in a single <literal>GROUP BY</literal>
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
clause, then the final list of grouping sets is the cross product of the
individual items. For example:
<programlisting>
2015-11-17 00:59:55 +01:00
GROUP BY a, CUBE (b, c), GROUPING SETS ((d), (e))
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
</programlisting>
is equivalent to
<programlisting>
GROUP BY GROUPING SETS (
2015-11-17 00:59:55 +01:00
(a, b, c, d), (a, b, c, e),
(a, b, d), (a, b, e),
(a, c, d), (a, c, e),
(a, d), (a, e)
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
)
</programlisting>
</para>
Implement GROUP BY DISTINCT
With grouping sets, it's possible that some of the grouping sets are
duplicate. This is especially common with CUBE and ROLLUP clauses. For
example GROUP BY CUBE (a,b), CUBE (b,c) is equivalent to
GROUP BY GROUPING SETS (
(a, b, c),
(a, b, c),
(a, b, c),
(a, b),
(a, b),
(a, b),
(a),
(a),
(a),
(c, a),
(c, a),
(c, a),
(c),
(b, c),
(b),
()
)
Some of the grouping sets are calculated multiple times, which is mostly
unnecessary. This commit implements a new GROUP BY DISTINCT feature, as
defined in the SQL standard, which eliminates the duplicate sets.
Author: Vik Fearing
Reviewed-by: Erik Rijkers, Georgios Kokolatos, Tomas Vondra
Discussion: https://postgr.es/m/bf3805a8-d7d1-ae61-fece-761b7ff41ecc@postgresfriends.org
2021-03-18 17:45:38 +01:00
<para>
<indexterm zone="queries-grouping-sets">
<primary>ALL</primary>
<secondary>GROUP BY ALL</secondary>
</indexterm>
<indexterm zone="queries-grouping-sets">
<primary>DISTINCT</primary>
<secondary>GROUP BY DISTINCT</secondary>
</indexterm>
When specifying multiple grouping items together, the final set of grouping
sets might contain duplicates. For example:
<programlisting>
GROUP BY ROLLUP (a, b), ROLLUP (a, c)
</programlisting>
is equivalent to
<programlisting>
GROUP BY GROUPING SETS (
(a, b, c),
(a, b),
(a, b),
(a, c),
(a),
(a),
(a, c),
(a),
()
)
</programlisting>
If these duplicates are undesirable, they can be removed using the
<literal>DISTINCT</literal> clause directly on the <literal>GROUP BY</literal>.
Therefore:
<programlisting>
GROUP BY <emphasis>DISTINCT</emphasis> ROLLUP (a, b), ROLLUP (a, c)
</programlisting>
is equivalent to
<programlisting>
GROUP BY GROUPING SETS (
(a, b, c),
(a, b),
(a, c),
(a),
()
)
</programlisting>
This is not the same as using <literal>SELECT DISTINCT</literal> because the output
rows may still contain duplicates. If any of the ungrouped columns contains NULL,
it will be indistinguishable from the NULL used when that same column is grouped.
</para>
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
<note>
<para>
2017-10-09 03:44:17 +02:00
The construct <literal>(a, b)</literal> is normally recognized in expressions as
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
a <link linkend="sql-syntax-row-constructors">row constructor</link>.
2017-10-09 03:44:17 +02:00
Within the <literal>GROUP BY</literal> clause, this does not apply at the top
levels of expressions, and <literal>(a, b)</literal> is parsed as a list of
expressions as described above. If for some reason you <emphasis>need</emphasis>
a row constructor in a grouping expression, use <literal>ROW(a, b)</literal>.
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
</para>
</note>
</sect2>
2008-12-28 19:54:01 +01:00
<sect2 id="queries-window">
2011-01-29 19:00:18 +01:00
<title>Window Function Processing</title>
2008-12-28 19:54:01 +01:00
<indexterm zone="queries-window">
<primary>window function</primary>
2017-10-09 03:44:17 +02:00
<secondary>order of execution</secondary>
2008-12-28 19:54:01 +01:00
</indexterm>
<para>
If the query contains any window functions (see
2017-11-23 15:39:47 +01:00
<xref linkend="tutorial-window"/>,
<xref linkend="functions-window"/> and
<xref linkend="syntax-window-functions"/>), these functions are evaluated
2017-10-09 03:44:17 +02:00
after any grouping, aggregation, and <literal>HAVING</literal> filtering is
2008-12-28 19:54:01 +01:00
performed. That is, if the query uses any aggregates, <literal>GROUP
2017-10-09 03:44:17 +02:00
BY</literal>, or <literal>HAVING</literal>, then the rows seen by the window functions
2008-12-28 19:54:01 +01:00
are the group rows instead of the original table rows from
2017-10-09 03:44:17 +02:00
<literal>FROM</literal>/<literal>WHERE</literal>.
2008-12-28 19:54:01 +01:00
</para>
<para>
When multiple window functions are used, all the window functions having
2017-10-09 03:44:17 +02:00
syntactically equivalent <literal>PARTITION BY</literal> and <literal>ORDER BY</literal>
2008-12-28 19:54:01 +01:00
clauses in their window definitions are guaranteed to be evaluated in a
single pass over the data. Therefore they will see the same sort ordering,
2017-10-09 03:44:17 +02:00
even if the <literal>ORDER BY</literal> does not uniquely determine an ordering.
2008-12-28 19:54:01 +01:00
However, no guarantees are made about the evaluation of functions having
2017-10-09 03:44:17 +02:00
different <literal>PARTITION BY</literal> or <literal>ORDER BY</literal> specifications.
2008-12-28 19:54:01 +01:00
(In such cases a sort step is typically required between the passes of
window function evaluations, and the sort is not guaranteed to preserve
2017-10-09 03:44:17 +02:00
ordering of rows that its <literal>ORDER BY</literal> sees as equivalent.)
2008-12-28 19:54:01 +01:00
</para>
<para>
2008-12-31 01:08:39 +01:00
Currently, window functions always require presorted data, and so the
2008-12-28 19:54:01 +01:00
query output will be ordered according to one or another of the window
2017-10-09 03:44:17 +02:00
functions' <literal>PARTITION BY</literal>/<literal>ORDER BY</literal> clauses.
2012-08-14 18:36:35 +02:00
It is not recommended to rely on this, however. Use an explicit
2017-10-09 03:44:17 +02:00
top-level <literal>ORDER BY</literal> clause if you want to be sure the
2008-12-28 19:54:01 +01:00
results are sorted in a particular way.
</para>
2001-01-23 00:34:33 +01:00
</sect2>
</sect1>
<sect1 id="queries-select-lists">
<title>Select Lists</title>
2001-05-13 00:51:36 +02:00
<indexterm>
2003-08-31 19:32:24 +02:00
<primary>SELECT</primary>
2001-05-13 00:51:36 +02:00
<secondary>select list</secondary>
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
2001-03-25 00:03:26 +01:00
As shown in the previous section,
the table expression in the <command>SELECT</command> command
2001-01-23 00:34:33 +01:00
constructs an intermediate virtual table by possibly combining
tables, views, eliminating rows, grouping, etc. This table is
2001-03-25 00:03:26 +01:00
finally passed on to processing by the <firstterm>select list</firstterm>. The select
2001-01-23 00:34:33 +01:00
list determines which <emphasis>columns</emphasis> of the
2002-09-20 20:39:41 +02:00
intermediate table are actually output.
</para>
<sect2 id="queries-select-list-items">
2002-11-11 21:14:04 +01:00
<title>Select-List Items</title>
2002-09-20 20:39:41 +02:00
2003-08-31 19:32:24 +02:00
<indexterm>
<primary>*</primary>
</indexterm>
2002-09-20 20:39:41 +02:00
<para>
The simplest kind of select list is <literal>*</literal> which
emits all columns that the table expression produces. Otherwise,
a select list is a comma-separated list of value expressions (as
2017-11-23 15:39:47 +01:00
defined in <xref linkend="sql-expressions"/>). For instance, it
2002-09-20 20:39:41 +02:00
could be a list of column names:
2001-01-23 00:34:33 +01:00
<programlisting>
SELECT a, b, c FROM ...
</programlisting>
2017-10-09 03:44:17 +02:00
The columns names <literal>a</literal>, <literal>b</literal>, and <literal>c</literal>
2002-09-20 20:39:41 +02:00
are either the actual names of the columns of tables referenced
2017-10-09 03:44:17 +02:00
in the <literal>FROM</literal> clause, or the aliases given to them as
2017-11-23 15:39:47 +01:00
explained in <xref linkend="queries-table-aliases"/>. The name
2002-09-20 20:39:41 +02:00
space available in the select list is the same as in the
2017-10-09 03:44:17 +02:00
<literal>WHERE</literal> clause, unless grouping is used, in which case
it is the same as in the <literal>HAVING</literal> clause.
2002-09-20 20:39:41 +02:00
</para>
<para>
If more than one table has a column of the same name, the table
2007-02-01 01:28:19 +01:00
name must also be given, as in:
2001-01-23 00:34:33 +01:00
<programlisting>
2003-03-13 02:30:29 +01:00
SELECT tbl1.a, tbl2.a, tbl1.b FROM ...
2003-09-13 00:17:24 +02:00
</programlisting>
When working with multiple tables, it can also be useful to ask for
all the columns of a particular table:
<programlisting>
SELECT tbl1.*, tbl2.a FROM ...
2001-01-23 00:34:33 +01:00
</programlisting>
2017-11-23 15:39:47 +01:00
See <xref linkend="rowtypes-usage"/> for more about
2017-10-09 03:44:17 +02:00
the <replaceable>table_name</replaceable><literal>.*</literal> notation.
2002-09-20 20:39:41 +02:00
</para>
2001-01-23 00:34:33 +01:00
2002-09-20 20:39:41 +02:00
<para>
If an arbitrary value expression is used in the select list, it
conceptually adds a new virtual column to the returned table. The
2003-03-13 02:30:29 +01:00
value expression is evaluated once for each result row, with
2002-09-20 20:39:41 +02:00
the row's values substituted for any column references. But the
expressions in the select list do not have to reference any
2017-10-09 03:44:17 +02:00
columns in the table expression of the <literal>FROM</literal> clause;
2009-06-17 23:58:49 +02:00
they can be constant arithmetic expressions, for instance.
2002-09-20 20:39:41 +02:00
</para>
</sect2>
2001-01-23 00:34:33 +01:00
<sect2 id="queries-column-labels">
<title>Column Labels</title>
2001-05-13 00:51:36 +02:00
<indexterm zone="queries-column-labels">
2003-08-31 19:32:24 +02:00
<primary>alias</primary>
<secondary>in the select list</secondary>
2001-05-13 00:51:36 +02:00
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
2009-06-17 23:58:49 +02:00
The entries in the select list can be assigned names for subsequent
2017-10-09 03:44:17 +02:00
processing, such as for use in an <literal>ORDER BY</literal> clause
2009-04-27 18:27:36 +02:00
or for display by the client application. For example:
2001-01-23 00:34:33 +01:00
<programlisting>
SELECT a AS value, b + c AS sum FROM ...
</programlisting>
</para>
<para>
2017-10-09 03:44:17 +02:00
If no output column name is specified using <literal>AS</literal>,
2008-02-15 23:17:06 +01:00
the system assigns a default column name. For simple column references,
2008-10-04 23:56:55 +02:00
this is the name of the referenced column. For function
2001-01-23 00:34:33 +01:00
calls, this is the name of the function. For complex expressions,
the system will generate a generic name.
</para>
2008-02-15 23:17:06 +01:00
<para>
Allow most keywords to be used as column labels without requiring AS.
Up to now, if you tried to omit "AS" before a column label in a SELECT
list, it would only work if the column label was an IDENT, that is not
any known keyword. This is rather unfriendly considering that we have
so many keywords and are constantly growing more. In the wake of commit
1ed6b8956 it's possible to improve matters quite a bit.
We'd originally tried to make this work by having some of the existing
keyword categories be allowed without AS, but that didn't work too well,
because each category contains a few special cases that don't work
without AS. Instead, invent an entirely orthogonal keyword property
"can be bare column label", and mark all keywords that way for which
we don't get shift/reduce errors by doing so.
It turns out that of our 450 current keywords, all but 39 can be made
bare column labels, improving the situation by over 90%. This number
might move around a little depending on future grammar work, but it's
a pretty nice improvement.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
2020-09-18 22:46:26 +02:00
The <literal>AS</literal> key word is usually optional, but in some
cases where the desired column name matches a
<productname>PostgreSQL</productname> key word, you must write
<literal>AS</literal> or double-quote the column name in order to
avoid ambiguity.
(<xref linkend="sql-keywords-appendix"/> shows which key words
require <literal>AS</literal> to be used as a column label.)
For example, <literal>FROM</literal> is one such key word, so this
does not work:
2008-02-15 23:17:06 +01:00
<programlisting>
Allow most keywords to be used as column labels without requiring AS.
Up to now, if you tried to omit "AS" before a column label in a SELECT
list, it would only work if the column label was an IDENT, that is not
any known keyword. This is rather unfriendly considering that we have
so many keywords and are constantly growing more. In the wake of commit
1ed6b8956 it's possible to improve matters quite a bit.
We'd originally tried to make this work by having some of the existing
keyword categories be allowed without AS, but that didn't work too well,
because each category contains a few special cases that don't work
without AS. Instead, invent an entirely orthogonal keyword property
"can be bare column label", and mark all keywords that way for which
we don't get shift/reduce errors by doing so.
It turns out that of our 450 current keywords, all but 39 can be made
bare column labels, improving the situation by over 90%. This number
might move around a little depending on future grammar work, but it's
a pretty nice improvement.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
2020-09-18 22:46:26 +02:00
SELECT a from, b + c AS sum FROM ...
2008-02-15 23:17:06 +01:00
</programlisting>
Allow most keywords to be used as column labels without requiring AS.
Up to now, if you tried to omit "AS" before a column label in a SELECT
list, it would only work if the column label was an IDENT, that is not
any known keyword. This is rather unfriendly considering that we have
so many keywords and are constantly growing more. In the wake of commit
1ed6b8956 it's possible to improve matters quite a bit.
We'd originally tried to make this work by having some of the existing
keyword categories be allowed without AS, but that didn't work too well,
because each category contains a few special cases that don't work
without AS. Instead, invent an entirely orthogonal keyword property
"can be bare column label", and mark all keywords that way for which
we don't get shift/reduce errors by doing so.
It turns out that of our 450 current keywords, all but 39 can be made
bare column labels, improving the situation by over 90%. This number
might move around a little depending on future grammar work, but it's
a pretty nice improvement.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
2020-09-18 22:46:26 +02:00
but either of these do:
2008-02-15 23:17:06 +01:00
<programlisting>
Allow most keywords to be used as column labels without requiring AS.
Up to now, if you tried to omit "AS" before a column label in a SELECT
list, it would only work if the column label was an IDENT, that is not
any known keyword. This is rather unfriendly considering that we have
so many keywords and are constantly growing more. In the wake of commit
1ed6b8956 it's possible to improve matters quite a bit.
We'd originally tried to make this work by having some of the existing
keyword categories be allowed without AS, but that didn't work too well,
because each category contains a few special cases that don't work
without AS. Instead, invent an entirely orthogonal keyword property
"can be bare column label", and mark all keywords that way for which
we don't get shift/reduce errors by doing so.
It turns out that of our 450 current keywords, all but 39 can be made
bare column labels, improving the situation by over 90%. This number
might move around a little depending on future grammar work, but it's
a pretty nice improvement.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
2020-09-18 22:46:26 +02:00
SELECT a AS from, b + c AS sum FROM ...
SELECT a "from", b + c AS sum FROM ...
2008-02-15 23:17:06 +01:00
</programlisting>
Allow most keywords to be used as column labels without requiring AS.
Up to now, if you tried to omit "AS" before a column label in a SELECT
list, it would only work if the column label was an IDENT, that is not
any known keyword. This is rather unfriendly considering that we have
so many keywords and are constantly growing more. In the wake of commit
1ed6b8956 it's possible to improve matters quite a bit.
We'd originally tried to make this work by having some of the existing
keyword categories be allowed without AS, but that didn't work too well,
because each category contains a few special cases that don't work
without AS. Instead, invent an entirely orthogonal keyword property
"can be bare column label", and mark all keywords that way for which
we don't get shift/reduce errors by doing so.
It turns out that of our 450 current keywords, all but 39 can be made
bare column labels, improving the situation by over 90%. This number
might move around a little depending on future grammar work, but it's
a pretty nice improvement.
Mark Dilger, based on work by myself and Robert Haas;
review by John Naylor
Discussion: https://postgr.es/m/38ca86db-42ab-9b48-2902-337a0d6b8311@2ndquadrant.com
2020-09-18 22:46:26 +02:00
For greatest safety against possible
future key word additions, it is recommended that you always either
2008-02-15 23:17:06 +01:00
write <literal>AS</literal> or double-quote the output column name.
</para>
2001-01-23 00:34:33 +01:00
<note>
<para>
The naming of output columns here is different from that done in
2017-10-09 03:44:17 +02:00
the <literal>FROM</literal> clause (see <xref
2017-11-23 15:39:47 +01:00
linkend="queries-table-aliases"/>). It is possible
2009-06-17 23:58:49 +02:00
to rename the same column twice, but the name assigned in
2002-09-20 20:39:41 +02:00
the select list is the one that will be passed on.
2001-01-23 00:34:33 +01:00
</para>
</note>
</sect2>
<sect2 id="queries-distinct">
2003-11-04 10:55:39 +01:00
<title><literal>DISTINCT</literal></title>
2001-01-23 00:34:33 +01:00
Implement GROUP BY DISTINCT
With grouping sets, it's possible that some of the grouping sets are
duplicate. This is especially common with CUBE and ROLLUP clauses. For
example GROUP BY CUBE (a,b), CUBE (b,c) is equivalent to
GROUP BY GROUPING SETS (
(a, b, c),
(a, b, c),
(a, b, c),
(a, b),
(a, b),
(a, b),
(a),
(a),
(a),
(c, a),
(c, a),
(c, a),
(c),
(b, c),
(b),
()
)
Some of the grouping sets are calculated multiple times, which is mostly
unnecessary. This commit implements a new GROUP BY DISTINCT feature, as
defined in the SQL standard, which eliminates the duplicate sets.
Author: Vik Fearing
Reviewed-by: Erik Rijkers, Georgios Kokolatos, Tomas Vondra
Discussion: https://postgr.es/m/bf3805a8-d7d1-ae61-fece-761b7ff41ecc@postgresfriends.org
2021-03-18 17:45:38 +01:00
<indexterm zone="queries-distinct">
<primary>ALL</primary>
<secondary>SELECT ALL</secondary>
</indexterm>
2001-05-13 00:51:36 +02:00
<indexterm zone="queries-distinct">
2003-08-31 19:32:24 +02:00
<primary>DISTINCT</primary>
Implement GROUP BY DISTINCT
With grouping sets, it's possible that some of the grouping sets are
duplicate. This is especially common with CUBE and ROLLUP clauses. For
example GROUP BY CUBE (a,b), CUBE (b,c) is equivalent to
GROUP BY GROUPING SETS (
(a, b, c),
(a, b, c),
(a, b, c),
(a, b),
(a, b),
(a, b),
(a),
(a),
(a),
(c, a),
(c, a),
(c, a),
(c),
(b, c),
(b),
()
)
Some of the grouping sets are calculated multiple times, which is mostly
unnecessary. This commit implements a new GROUP BY DISTINCT feature, as
defined in the SQL standard, which eliminates the duplicate sets.
Author: Vik Fearing
Reviewed-by: Erik Rijkers, Georgios Kokolatos, Tomas Vondra
Discussion: https://postgr.es/m/bf3805a8-d7d1-ae61-fece-761b7ff41ecc@postgresfriends.org
2021-03-18 17:45:38 +01:00
<secondary>SELECT DISTINCT</secondary>
2003-08-31 19:32:24 +02:00
</indexterm>
<indexterm zone="queries-distinct">
<primary>duplicates</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
After the select list has been processed, the result table can
2004-12-23 06:37:40 +01:00
optionally be subject to the elimination of duplicate rows. The
<literal>DISTINCT</literal> key word is written directly after
<literal>SELECT</literal> to specify this:
2001-01-23 00:34:33 +01:00
<synopsis>
SELECT DISTINCT <replaceable>select_list</replaceable> ...
</synopsis>
2017-10-09 03:44:17 +02:00
(Instead of <literal>DISTINCT</literal> the key word <literal>ALL</literal>
2004-12-23 06:37:40 +01:00
can be used to specify the default behavior of retaining all rows.)
2001-01-23 00:34:33 +01:00
</para>
2011-04-08 17:36:05 +02:00
<indexterm>
2017-10-09 03:44:17 +02:00
<primary>null value</primary>
<secondary sortas="DISTINCT">in DISTINCT</secondary>
2011-04-08 17:36:05 +02:00
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
Obviously, two rows are considered distinct if they differ in at
2002-08-05 21:43:31 +02:00
least one column value. Null values are considered equal in this
2001-02-10 09:30:13 +01:00
comparison.
2001-01-23 00:34:33 +01:00
</para>
<para>
Alternatively, an arbitrary expression can determine what rows are
to be considered distinct:
<synopsis>
SELECT DISTINCT ON (<replaceable>expression</replaceable> <optional>, <replaceable>expression</replaceable> ...</optional>) <replaceable>select_list</replaceable> ...
</synopsis>
Here <replaceable>expression</replaceable> is an arbitrary value
expression that is evaluated for all rows. A set of rows for
2001-02-10 09:30:13 +01:00
which all the expressions are equal are considered duplicates, and
2002-09-20 20:39:41 +02:00
only the first row of the set is kept in the output. Note that
the <quote>first row</quote> of a set is unpredictable unless the
2001-02-10 09:30:13 +01:00
query is sorted on enough columns to guarantee a unique ordering
2017-10-09 03:44:17 +02:00
of the rows arriving at the <literal>DISTINCT</literal> filter.
(<literal>DISTINCT ON</literal> processing occurs after <literal>ORDER
BY</literal> sorting.)
2001-01-23 00:34:33 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
The <literal>DISTINCT ON</literal> clause is not part of the SQL standard
2002-09-20 20:39:41 +02:00
and is sometimes considered bad style because of the potentially
indeterminate nature of its results. With judicious use of
2017-10-09 03:44:17 +02:00
<literal>GROUP BY</literal> and subqueries in <literal>FROM</literal>, this
2002-09-20 20:39:41 +02:00
construct can be avoided, but it is often the most convenient
alternative.
2001-01-23 00:34:33 +01:00
</para>
</sect2>
</sect1>
2002-09-20 20:39:41 +02:00
2001-01-23 00:34:33 +01:00
<sect1 id="queries-union">
2020-11-02 11:51:46 +01:00
<title>Combining Queries (<literal>UNION</literal>, <literal>INTERSECT</literal>, <literal>EXCEPT</literal>)</title>
2001-01-23 00:34:33 +01:00
2001-05-13 00:51:36 +02:00
<indexterm zone="queries-union">
2003-08-31 19:32:24 +02:00
<primary>UNION</primary>
</indexterm>
<indexterm zone="queries-union">
<primary>INTERSECT</primary>
</indexterm>
<indexterm zone="queries-union">
<primary>EXCEPT</primary>
</indexterm>
<indexterm zone="queries-union">
<primary>set union</primary>
</indexterm>
<indexterm zone="queries-union">
<primary>set intersection</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
<indexterm zone="queries-union">
2003-08-31 19:32:24 +02:00
<primary>set difference</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
<indexterm zone="queries-union">
2003-08-31 19:32:24 +02:00
<primary>set operation</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
2001-01-23 00:34:33 +01:00
<para>
The results of two queries can be combined using the set operations
union, intersection, and difference. The syntax is
<synopsis>
<replaceable>query1</replaceable> UNION <optional>ALL</optional> <replaceable>query2</replaceable>
<replaceable>query1</replaceable> INTERSECT <optional>ALL</optional> <replaceable>query2</replaceable>
<replaceable>query1</replaceable> EXCEPT <optional>ALL</optional> <replaceable>query2</replaceable>
</synopsis>
2021-10-05 16:24:14 +02:00
where <replaceable>query1</replaceable> and
2001-01-23 00:34:33 +01:00
<replaceable>query2</replaceable> are queries that can use any of
2021-10-05 16:24:14 +02:00
the features discussed up to this point.
2001-01-23 00:34:33 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
<literal>UNION</literal> effectively appends the result of
2001-01-23 00:34:33 +01:00
<replaceable>query2</replaceable> to the result of
<replaceable>query1</replaceable> (although there is no guarantee
2001-02-10 09:30:13 +01:00
that this is the order in which the rows are actually returned).
2004-12-23 06:37:40 +01:00
Furthermore, it eliminates duplicate rows from its result, in the same
2017-10-09 03:44:17 +02:00
way as <literal>DISTINCT</literal>, unless <literal>UNION ALL</literal> is used.
2001-01-23 00:34:33 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
<literal>INTERSECT</literal> returns all rows that are both in the result
2002-09-20 20:39:41 +02:00
of <replaceable>query1</replaceable> and in the result of
2001-01-23 00:34:33 +01:00
<replaceable>query2</replaceable>. Duplicate rows are eliminated
2017-10-09 03:44:17 +02:00
unless <literal>INTERSECT ALL</literal> is used.
2001-01-23 00:34:33 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
<literal>EXCEPT</literal> returns all rows that are in the result of
2002-09-20 20:39:41 +02:00
<replaceable>query1</replaceable> but not in the result of
<replaceable>query2</replaceable>. (This is sometimes called the
2017-10-09 03:44:17 +02:00
<firstterm>difference</firstterm> between two queries.) Again, duplicates
are eliminated unless <literal>EXCEPT ALL</literal> is used.
2001-01-23 00:34:33 +01:00
</para>
<para>
In order to calculate the union, intersection, or difference of two
queries, the two queries must be <quote>union compatible</quote>,
2004-12-23 06:37:40 +01:00
which means that they return the same number of columns and
the corresponding columns have compatible data types, as
2017-11-23 15:39:47 +01:00
described in <xref linkend="typeconv-union-case"/>.
2001-01-23 00:34:33 +01:00
</para>
2021-10-05 16:24:14 +02:00
<para>
Set operations can be combined, for example
<synopsis>
<replaceable>query1</replaceable> UNION <replaceable>query2</replaceable> EXCEPT <replaceable>query3</replaceable>
</synopsis>
which is equivalent to
<synopsis>
(<replaceable>query1</replaceable> UNION <replaceable>query2</replaceable>) EXCEPT <replaceable>query3</replaceable>
</synopsis>
As shown here, you can use parentheses to control the order of
evaluation. Without parentheses, <literal>UNION</literal>
and <literal>EXCEPT</literal> associate left-to-right,
but <literal>INTERSECT</literal> binds more tightly than those two
operators. Thus
<synopsis>
<replaceable>query1</replaceable> UNION <replaceable>query2</replaceable> INTERSECT <replaceable>query3</replaceable>
</synopsis>
means
<synopsis>
<replaceable>query1</replaceable> UNION (<replaceable>query2</replaceable> INTERSECT <replaceable>query3</replaceable>)
</synopsis>
You can also surround an individual <replaceable>query</replaceable>
with parentheses. This is important if
the <replaceable>query</replaceable> needs to use any of the clauses
discussed in following sections, such as <literal>LIMIT</literal>.
Without parentheses, you'll get a syntax error, or else the clause will
be understood as applying to the output of the set operation rather
than one of its inputs. For example,
<synopsis>
SELECT a FROM b UNION SELECT x FROM y LIMIT 10
</synopsis>
is accepted, but it means
<synopsis>
(SELECT a FROM b UNION SELECT x FROM y) LIMIT 10
</synopsis>
not
<synopsis>
SELECT a FROM b UNION (SELECT x FROM y LIMIT 10)
</synopsis>
</para>
2001-01-23 00:34:33 +01:00
</sect1>
<sect1 id="queries-order">
2020-11-02 11:51:46 +01:00
<title>Sorting Rows (<literal>ORDER BY</literal>)</title>
2001-05-13 00:51:36 +02:00
<indexterm zone="queries-order">
<primary>sorting</primary>
2003-08-31 19:32:24 +02:00
</indexterm>
<indexterm zone="queries-order">
<primary>ORDER BY</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
2006-09-18 21:54:01 +02:00
2001-01-23 00:34:33 +01:00
<para>
After a query has produced an output table (after the select list
has been processed) it can optionally be sorted. If sorting is not
2004-12-23 06:37:40 +01:00
chosen, the rows will be returned in an unspecified order. The actual
2001-01-23 00:34:33 +01:00
order in that case will depend on the scan and join plan types and
the order on disk, but it must not be relied on. A particular
2001-02-10 09:30:13 +01:00
output ordering can only be guaranteed if the sort step is explicitly
2001-01-23 00:34:33 +01:00
chosen.
</para>
<para>
2017-10-09 03:44:17 +02:00
The <literal>ORDER BY</literal> clause specifies the sort order:
2001-01-23 00:34:33 +01:00
<synopsis>
2001-10-09 20:46:00 +02:00
SELECT <replaceable>select_list</replaceable>
FROM <replaceable>table_expression</replaceable>
2007-01-09 03:14:16 +01:00
ORDER BY <replaceable>sort_expression1</replaceable> <optional>ASC | DESC</optional> <optional>NULLS { FIRST | LAST }</optional>
<optional>, <replaceable>sort_expression2</replaceable> <optional>ASC | DESC</optional> <optional>NULLS { FIRST | LAST }</optional> ...</optional>
2001-01-23 00:34:33 +01:00
</synopsis>
2006-10-24 04:24:27 +02:00
The sort expression(s) can be any expression that would be valid in the
2007-02-01 01:28:19 +01:00
query's select list. An example is:
2001-01-23 00:34:33 +01:00
<programlisting>
2006-10-24 04:24:27 +02:00
SELECT a, b FROM table1 ORDER BY a + b, c;
2001-01-23 00:34:33 +01:00
</programlisting>
2006-10-24 04:24:27 +02:00
When more than one expression is specified,
the later values are used to sort rows that are equal according to the
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
earlier values. Each expression can be followed by an optional
2017-10-09 03:44:17 +02:00
<literal>ASC</literal> or <literal>DESC</literal> keyword to set the sort direction to
ascending or descending. <literal>ASC</literal> order is the default.
2002-09-20 20:39:41 +02:00
Ascending order puts smaller values first, where
<quote>smaller</quote> is defined in terms of the
<literal><</literal> operator. Similarly, descending order is
determined with the <literal>></literal> operator.
2003-09-13 00:17:24 +02:00
<footnote>
<para>
2017-10-09 03:44:17 +02:00
Actually, <productname>PostgreSQL</productname> uses the <firstterm>default B-tree
operator class</firstterm> for the expression's data type to determine the sort
ordering for <literal>ASC</literal> and <literal>DESC</literal>. Conventionally,
2003-11-01 02:56:29 +01:00
data types will be set up so that the <literal><</literal> and
2003-09-13 00:17:24 +02:00
<literal>></literal> operators correspond to this sort ordering,
2003-11-01 02:56:29 +01:00
but a user-defined data type's designer could choose to do something
2003-09-13 00:17:24 +02:00
different.
</para>
</footnote>
2001-01-23 00:34:33 +01:00
</para>
2007-01-09 03:14:16 +01:00
<para>
2017-10-09 03:44:17 +02:00
The <literal>NULLS FIRST</literal> and <literal>NULLS LAST</literal> options can be
2007-01-09 03:14:16 +01:00
used to determine whether nulls appear before or after non-null values
2009-06-17 23:58:49 +02:00
in the sort ordering. By default, null values sort as if larger than any
2017-10-09 03:44:17 +02:00
non-null value; that is, <literal>NULLS FIRST</literal> is the default for
<literal>DESC</literal> order, and <literal>NULLS LAST</literal> otherwise.
2007-01-09 03:14:16 +01:00
</para>
2007-01-09 17:59:20 +01:00
<para>
Note that the ordering options are considered independently for each
2017-10-09 03:44:17 +02:00
sort column. For example <literal>ORDER BY x, y DESC</literal> means
<literal>ORDER BY x ASC, y DESC</literal>, which is not the same as
<literal>ORDER BY x DESC, y DESC</literal>.
2007-01-09 17:59:20 +01:00
</para>
2001-01-23 00:34:33 +01:00
<para>
2017-10-09 03:44:17 +02:00
A <replaceable>sort_expression</replaceable> can also be the column label or number
2007-02-01 01:28:19 +01:00
of an output column, as in:
2006-10-24 04:24:27 +02:00
<programlisting>
SELECT a + b AS sum, c FROM table1 ORDER BY sum;
SELECT a, max(b) FROM table1 GROUP BY a ORDER BY 1;
</programlisting>
both of which sort by the first output column. Note that an output
2009-06-17 23:58:49 +02:00
column name has to stand alone, that is, it cannot be used in an expression
2017-10-09 03:44:17 +02:00
— for example, this is <emphasis>not</emphasis> correct:
2006-10-24 04:24:27 +02:00
<programlisting>
SELECT a + b AS sum, c FROM table1 ORDER BY sum + c; -- wrong
</programlisting>
2008-10-04 23:56:55 +02:00
This restriction is made to reduce ambiguity. There is still
2017-10-09 03:44:17 +02:00
ambiguity if an <literal>ORDER BY</literal> item is a simple name that
2006-10-24 04:24:27 +02:00
could match either an output column name or a column from the table
expression. The output column is used in such cases. This would
2017-10-09 03:44:17 +02:00
only cause confusion if you use <literal>AS</literal> to rename an output
2006-10-24 04:24:27 +02:00
column to match some other table column's name.
</para>
<para>
2017-10-09 03:44:17 +02:00
<literal>ORDER BY</literal> can be applied to the result of a
<literal>UNION</literal>, <literal>INTERSECT</literal>, or <literal>EXCEPT</literal>
2006-10-24 04:24:27 +02:00
combination, but in this case it is only permitted to sort by
output column names or numbers, not by expressions.
2001-01-23 00:34:33 +01:00
</para>
</sect1>
2002-09-20 20:39:41 +02:00
2001-01-23 00:34:33 +01:00
<sect1 id="queries-limit">
2003-11-04 10:55:39 +01:00
<title><literal>LIMIT</literal> and <literal>OFFSET</literal></title>
2001-01-23 00:34:33 +01:00
2001-05-13 00:51:36 +02:00
<indexterm zone="queries-limit">
2003-08-31 19:32:24 +02:00
<primary>LIMIT</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
<indexterm zone="queries-limit">
2003-08-31 19:32:24 +02:00
<primary>OFFSET</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
2002-09-20 20:39:41 +02:00
<para>
2017-10-09 03:44:17 +02:00
<literal>LIMIT</literal> and <literal>OFFSET</literal> allow you to retrieve just
2002-09-20 20:39:41 +02:00
a portion of the rows that are generated by the rest of the query:
2001-01-23 00:34:33 +01:00
<synopsis>
2001-10-09 20:46:00 +02:00
SELECT <replaceable>select_list</replaceable>
FROM <replaceable>table_expression</replaceable>
2007-01-09 03:14:16 +01:00
<optional> ORDER BY ... </optional>
2006-02-19 00:14:45 +01:00
<optional> LIMIT { <replaceable>number</replaceable> | ALL } </optional> <optional> OFFSET <replaceable>number</replaceable> </optional>
2001-01-23 00:34:33 +01:00
</synopsis>
2002-09-20 20:39:41 +02:00
</para>
2001-01-23 00:34:33 +01:00
<para>
2002-09-20 20:39:41 +02:00
If a limit count is given, no more than that many rows will be
2016-04-14 16:57:29 +02:00
returned (but possibly fewer, if the query itself yields fewer rows).
2017-10-09 03:44:17 +02:00
<literal>LIMIT ALL</literal> is the same as omitting the <literal>LIMIT</literal>
clause, as is <literal>LIMIT</literal> with a NULL argument.
2001-11-19 04:58:25 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
<literal>OFFSET</literal> says to skip that many rows before beginning to
return rows. <literal>OFFSET 0</literal> is the same as omitting the
<literal>OFFSET</literal> clause, as is <literal>OFFSET</literal> with a NULL argument.
2016-04-14 16:57:29 +02:00
</para>
<para>
2017-10-09 03:44:17 +02:00
If both <literal>OFFSET</literal>
and <literal>LIMIT</literal> appear, then <literal>OFFSET</literal> rows are
skipped before starting to count the <literal>LIMIT</literal> rows that
2002-09-20 20:39:41 +02:00
are returned.
2001-01-23 00:34:33 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
When using <literal>LIMIT</literal>, it is important to use an
<literal>ORDER BY</literal> clause that constrains the result rows into a
2002-09-20 20:39:41 +02:00
unique order. Otherwise you will get an unpredictable subset of
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
the query's rows. You might be asking for the tenth through
2009-06-17 23:58:49 +02:00
twentieth rows, but tenth through twentieth in what ordering? The
2017-10-09 03:44:17 +02:00
ordering is unknown, unless you specified <literal>ORDER BY</literal>.
2001-01-23 00:34:33 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
The query optimizer takes <literal>LIMIT</literal> into account when
2009-04-27 18:27:36 +02:00
generating query plans, so you are very likely to get different
2002-09-20 20:39:41 +02:00
plans (yielding different row orders) depending on what you give
2017-10-09 03:44:17 +02:00
for <literal>LIMIT</literal> and <literal>OFFSET</literal>. Thus, using
different <literal>LIMIT</literal>/<literal>OFFSET</literal> values to select
2001-01-23 00:34:33 +01:00
different subsets of a query result <emphasis>will give
inconsistent results</emphasis> unless you enforce a predictable
2017-10-09 03:44:17 +02:00
result ordering with <literal>ORDER BY</literal>. This is not a bug; it
2002-09-20 20:39:41 +02:00
is an inherent consequence of the fact that SQL does not promise to
deliver the results of a query in any particular order unless
2017-10-09 03:44:17 +02:00
<literal>ORDER BY</literal> is used to constrain the order.
2001-01-23 00:34:33 +01:00
</para>
2004-12-23 06:37:40 +01:00
<para>
2017-10-09 03:44:17 +02:00
The rows skipped by an <literal>OFFSET</literal> clause still have to be
computed inside the server; therefore a large <literal>OFFSET</literal>
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
might be inefficient.
2004-12-23 06:37:40 +01:00
</para>
2001-01-23 00:34:33 +01:00
</sect1>
2006-09-18 21:54:01 +02:00
<sect1 id="queries-values">
<title><literal>VALUES</literal> Lists</title>
<indexterm zone="queries-values">
<primary>VALUES</primary>
</indexterm>
<para>
2017-10-09 03:44:17 +02:00
<literal>VALUES</literal> provides a way to generate a <quote>constant table</quote>
2006-09-18 21:54:01 +02:00
that can be used in a query without having to actually create and populate
a table on-disk. The syntax is
<synopsis>
2017-10-09 04:00:57 +02:00
VALUES ( <replaceable class="parameter">expression</replaceable> [, ...] ) [, ...]
2006-09-18 21:54:01 +02:00
</synopsis>
2009-06-17 23:58:49 +02:00
Each parenthesized list of expressions generates a row in the table.
2006-09-18 21:54:01 +02:00
The lists must all have the same number of elements (i.e., the number
of columns in the table), and corresponding entries in each list must
2006-10-23 20:10:32 +02:00
have compatible data types. The actual data type assigned to each column
2017-10-09 03:44:17 +02:00
of the result is determined using the same rules as for <literal>UNION</literal>
2017-11-23 15:39:47 +01:00
(see <xref linkend="typeconv-union-case"/>).
2006-09-18 21:54:01 +02:00
</para>
<para>
2007-02-01 01:28:19 +01:00
As an example:
2006-09-18 21:54:01 +02:00
<programlisting>
VALUES (1, 'one'), (2, 'two'), (3, 'three');
</programlisting>
will return a table of two columns and three rows. It's effectively
2007-02-01 01:28:19 +01:00
equivalent to:
2006-09-18 21:54:01 +02:00
<programlisting>
SELECT 1 AS column1, 'one' AS column2
UNION ALL
SELECT 2, 'two'
UNION ALL
SELECT 3, 'three';
</programlisting>
By default, <productname>PostgreSQL</productname> assigns the names
2017-10-09 03:44:17 +02:00
<literal>column1</literal>, <literal>column2</literal>, etc. to the columns of a
<literal>VALUES</literal> table. The column names are not specified by the
2006-09-18 21:54:01 +02:00
SQL standard and different database systems do it differently, so
it's usually better to override the default names with a table alias
2013-06-07 03:03:04 +02:00
list, like this:
<programlisting>
2015-08-06 05:03:45 +02:00
=> SELECT * FROM (VALUES (1, 'one'), (2, 'two'), (3, 'three')) AS t (num,letter);
2013-06-07 03:03:04 +02:00
num | letter
-----+--------
1 | one
2 | two
3 | three
(3 rows)
</programlisting>
2006-09-18 21:54:01 +02:00
</para>
<para>
2017-10-09 03:44:17 +02:00
Syntactically, <literal>VALUES</literal> followed by expression lists is
2009-04-27 18:27:36 +02:00
treated as equivalent to:
2006-09-18 21:54:01 +02:00
<synopsis>
SELECT <replaceable>select_list</replaceable> FROM <replaceable>table_expression</replaceable>
</synopsis>
2017-10-09 03:44:17 +02:00
and can appear anywhere a <literal>SELECT</literal> can. For example, you can
use it as part of a <literal>UNION</literal>, or attach a
<replaceable>sort_specification</replaceable> (<literal>ORDER BY</literal>,
<literal>LIMIT</literal>, and/or <literal>OFFSET</literal>) to it. <literal>VALUES</literal>
is most commonly used as the data source in an <command>INSERT</command> command,
2006-09-18 21:54:01 +02:00
and next most commonly as a subquery.
</para>
<para>
2017-11-23 15:39:47 +01:00
For more information see <xref linkend="sql-values"/>.
2006-09-18 21:54:01 +02:00
</para>
</sect1>
2008-10-04 23:56:55 +02:00
<sect1 id="queries-with">
2011-01-20 03:15:45 +01:00
<title><literal>WITH</literal> Queries (Common Table Expressions)</title>
2008-10-04 23:56:55 +02:00
<indexterm zone="queries-with">
<primary>WITH</primary>
<secondary>in SELECT</secondary>
</indexterm>
<indexterm>
<primary>common table expression</primary>
<see>WITH</see>
</indexterm>
<para>
2017-10-09 03:44:17 +02:00
<literal>WITH</literal> provides a way to write auxiliary statements for use in a
2011-02-28 07:17:18 +01:00
larger query. These statements, which are often referred to as Common
Table Expressions or <acronym>CTE</acronym>s, can be thought of as defining
temporary tables that exist just for one query. Each auxiliary statement
2017-10-09 03:44:17 +02:00
in a <literal>WITH</literal> clause can be a <command>SELECT</command>,
<command>INSERT</command>, <command>UPDATE</command>, or <command>DELETE</command>; and the
<literal>WITH</literal> clause itself is attached to a primary statement that can
also be a <command>SELECT</command>, <command>INSERT</command>, <command>UPDATE</command>, or
<command>DELETE</command>.
2011-02-28 07:17:18 +01:00
</para>
<sect2 id="queries-with-select">
2017-10-09 03:44:17 +02:00
<title><command>SELECT</command> in <literal>WITH</literal></title>
2011-02-28 07:17:18 +01:00
<para>
2017-10-09 03:44:17 +02:00
The basic value of <command>SELECT</command> in <literal>WITH</literal> is to
2011-02-28 07:17:18 +01:00
break down complicated queries into simpler parts. An example is:
2008-10-04 23:56:55 +02:00
<programlisting>
WITH regional_sales AS (
2018-04-05 02:57:26 +02:00
SELECT region, SUM(amount) AS total_sales
FROM orders
GROUP BY region
), top_regions AS (
SELECT region
FROM regional_sales
WHERE total_sales > (SELECT SUM(total_sales)/10 FROM regional_sales)
)
2008-10-04 23:56:55 +02:00
SELECT region,
product,
SUM(quantity) AS product_units,
SUM(amount) AS product_sales
FROM orders
WHERE region IN (SELECT region FROM top_regions)
GROUP BY region, product;
</programlisting>
which displays per-product sales totals in only the top sales regions.
2017-10-09 03:44:17 +02:00
The <literal>WITH</literal> clause defines two auxiliary statements named
<structname>regional_sales</structname> and <structname>top_regions</structname>,
where the output of <structname>regional_sales</structname> is used in
<structname>top_regions</structname> and the output of <structname>top_regions</structname>
is used in the primary <command>SELECT</command> query.
This example could have been written without <literal>WITH</literal>,
2011-05-19 00:14:45 +02:00
but we'd have needed two levels of nested sub-<command>SELECT</command>s. It's a bit
2008-10-04 23:56:55 +02:00
easier to follow this way.
</para>
2020-10-13 06:29:06 +02:00
</sect2>
<sect2 id="queries-with-recursive">
<title>Recursive Queries</title>
2008-10-04 23:56:55 +02:00
<para>
2018-08-09 22:19:32 +02:00
<indexterm>
<primary>RECURSIVE</primary>
<secondary>in common table expressions</secondary>
</indexterm>
2017-10-09 03:44:17 +02:00
The optional <literal>RECURSIVE</literal> modifier changes <literal>WITH</literal>
2008-10-04 23:56:55 +02:00
from a mere syntactic convenience into a feature that accomplishes
things not otherwise possible in standard SQL. Using
2017-10-09 03:44:17 +02:00
<literal>RECURSIVE</literal>, a <literal>WITH</literal> query can refer to its own
2008-10-04 23:56:55 +02:00
output. A very simple example is this query to sum the integers from 1
through 100:
<programlisting>
WITH RECURSIVE t(n) AS (
VALUES (1)
UNION ALL
SELECT n+1 FROM t WHERE n < 100
)
SELECT sum(n) FROM t;
</programlisting>
2017-10-09 03:44:17 +02:00
The general form of a recursive <literal>WITH</literal> query is always a
<firstterm>non-recursive term</firstterm>, then <literal>UNION</literal> (or
<literal>UNION ALL</literal>), then a
<firstterm>recursive term</firstterm>, where only the recursive term can contain
2008-10-04 23:56:55 +02:00
a reference to the query's own output. Such a query is executed as
follows:
</para>
<procedure>
<title>Recursive Query Evaluation</title>
<step performance="required">
<para>
2017-10-09 03:44:17 +02:00
Evaluate the non-recursive term. For <literal>UNION</literal> (but not
<literal>UNION ALL</literal>), discard duplicate rows. Include all remaining
2008-10-07 21:27:04 +02:00
rows in the result of the recursive query, and also place them in a
2017-10-09 03:44:17 +02:00
temporary <firstterm>working table</firstterm>.
2008-10-04 23:56:55 +02:00
</para>
</step>
<step performance="required">
<para>
So long as the working table is not empty, repeat these steps:
</para>
<substeps>
<step performance="required">
<para>
Evaluate the recursive term, substituting the current contents of
2008-10-07 21:27:04 +02:00
the working table for the recursive self-reference.
2017-10-09 03:44:17 +02:00
For <literal>UNION</literal> (but not <literal>UNION ALL</literal>), discard
2008-10-07 21:27:04 +02:00
duplicate rows and rows that duplicate any previous result row.
Include all remaining rows in the result of the recursive query, and
2017-10-09 03:44:17 +02:00
also place them in a temporary <firstterm>intermediate table</firstterm>.
2008-10-04 23:56:55 +02:00
</para>
</step>
<step performance="required">
<para>
Replace the contents of the working table with the contents of the
intermediate table, then empty the intermediate table.
</para>
</step>
</substeps>
</step>
</procedure>
<note>
<para>
Strictly speaking, this process is iteration not recursion, but
2017-10-09 03:44:17 +02:00
<literal>RECURSIVE</literal> is the terminology chosen by the SQL standards
2008-10-04 23:56:55 +02:00
committee.
</para>
</note>
<para>
In the example above, the working table has just a single row in each step,
and it takes on the values from 1 through 100 in successive steps. In
2017-10-09 03:44:17 +02:00
the 100th step, there is no output because of the <literal>WHERE</literal>
2008-10-04 23:56:55 +02:00
clause, and so the query terminates.
</para>
<para>
Recursive queries are typically used to deal with hierarchical or
tree-structured data. A useful example is this query to find all the
direct and indirect sub-parts of a product, given only a table that
shows immediate inclusions:
<programlisting>
WITH RECURSIVE included_parts(sub_part, part, quantity) AS (
SELECT sub_part, part, quantity FROM parts WHERE part = 'our_product'
UNION ALL
SELECT p.sub_part, p.part, p.quantity
FROM included_parts pr, parts p
WHERE p.part = pr.sub_part
2018-04-05 02:57:26 +02:00
)
2008-10-04 23:56:55 +02:00
SELECT sub_part, SUM(quantity) as total_quantity
FROM included_parts
GROUP BY sub_part
</programlisting>
</para>
2020-10-13 06:29:06 +02:00
<sect3 id="queries-with-search">
<title>Search Order</title>
<para>
When computing a tree traversal using a recursive query, you might want to
order the results in either depth-first or breadth-first order. This can
be done by computing an ordering column alongside the other data columns
and using that to sort the results at the end. Note that this does not
actually control in which order the query evaluation visits the rows; that
is as always in SQL implementation-dependent. This approach merely
provides a convenient way to order the results afterwards.
</para>
<para>
To create a depth-first order, we compute for each result row an array of
rows that we have visited so far. For example, consider the following
query that searches a table <structname>tree</structname> using a
<structfield>link</structfield> field:
<programlisting>
WITH RECURSIVE search_tree(id, link, data) AS (
SELECT t.id, t.link, t.data
FROM tree t
UNION ALL
SELECT t.id, t.link, t.data
FROM tree t, search_tree st
WHERE t.id = st.link
)
SELECT * FROM search_tree;
</programlisting>
To add depth-first ordering information, you can write this:
<programlisting>
WITH RECURSIVE search_tree(id, link, data, <emphasis>path</emphasis>) AS (
SELECT t.id, t.link, t.data, <emphasis>ARRAY[t.id]</emphasis>
FROM tree t
UNION ALL
SELECT t.id, t.link, t.data, <emphasis>path || t.id</emphasis>
FROM tree t, search_tree st
WHERE t.id = st.link
)
SELECT * FROM search_tree <emphasis>ORDER BY path</emphasis>;
</programlisting>
</para>
<para>
In the general case where more than one field needs to be used to identify
a row, use an array of rows. For example, if we needed to track fields
<structfield>f1</structfield> and <structfield>f2</structfield>:
<programlisting>
WITH RECURSIVE search_tree(id, link, data, <emphasis>path</emphasis>) AS (
SELECT t.id, t.link, t.data, <emphasis>ARRAY[ROW(t.f1, t.f2)]</emphasis>
FROM tree t
UNION ALL
SELECT t.id, t.link, t.data, <emphasis>path || ROW(t.f1, t.f2)</emphasis>
FROM tree t, search_tree st
WHERE t.id = st.link
)
SELECT * FROM search_tree <emphasis>ORDER BY path</emphasis>;
</programlisting>
</para>
<tip>
<para>
Omit the <literal>ROW()</literal> syntax in the common case where only one
field needs to be tracked. This allows a simple array rather than a
composite-type array to be used, gaining efficiency.
</para>
</tip>
<para>
To create a breadth-first order, you can add a column that tracks the depth
of the search, for example:
<programlisting>
WITH RECURSIVE search_tree(id, link, data, <emphasis>depth</emphasis>) AS (
SELECT t.id, t.link, t.data, <emphasis>0</emphasis>
FROM tree t
UNION ALL
SELECT t.id, t.link, t.data, <emphasis>depth + 1</emphasis>
FROM tree t, search_tree st
WHERE t.id = st.link
)
SELECT * FROM search_tree <emphasis>ORDER BY depth</emphasis>;
</programlisting>
To get a stable sort, add data columns as secondary sorting columns.
</para>
<tip>
<para>
The recursive query evaluation algorithm produces its output in
breadth-first search order. However, this is an implementation detail and
it is perhaps unsound to rely on it. The order of the rows within each
level is certainly undefined, so some explicit ordering might be desired
in any case.
</para>
</tip>
2021-02-01 13:54:59 +01:00
<para>
There is built-in syntax to compute a depth- or breadth-first sort column.
For example:
<programlisting>
WITH RECURSIVE search_tree(id, link, data) AS (
SELECT t.id, t.link, t.data
FROM tree t
UNION ALL
SELECT t.id, t.link, t.data
FROM tree t, search_tree st
WHERE t.id = st.link
) <emphasis>SEARCH DEPTH FIRST BY id SET ordercol</emphasis>
SELECT * FROM search_tree ORDER BY ordercol;
WITH RECURSIVE search_tree(id, link, data) AS (
SELECT t.id, t.link, t.data
FROM tree t
UNION ALL
SELECT t.id, t.link, t.data
FROM tree t, search_tree st
WHERE t.id = st.link
) <emphasis>SEARCH BREADTH FIRST BY id SET ordercol</emphasis>
SELECT * FROM search_tree ORDER BY ordercol;
</programlisting>
This syntax is internally expanded to something similar to the above
hand-written forms. The <literal>SEARCH</literal> clause specifies whether
depth- or breadth first search is wanted, the list of columns to track for
sorting, and a column name that will contain the result data that can be
used for sorting. That column will implicitly be added to the output rows
of the CTE.
</para>
2020-10-13 06:29:06 +02:00
</sect3>
<sect3 id="queries-with-cycle">
<title>Cycle Detection</title>
2008-10-04 23:56:55 +02:00
<para>
When working with recursive queries it is important to be sure that
the recursive part of the query will eventually return no tuples,
2008-10-07 21:27:04 +02:00
or else the query will loop indefinitely. Sometimes, using
2017-10-09 03:44:17 +02:00
<literal>UNION</literal> instead of <literal>UNION ALL</literal> can accomplish this
2008-10-13 18:25:20 +02:00
by discarding rows that duplicate previous output rows. However, often a
cycle does not involve output rows that are completely duplicate: it may be
necessary to check just one or a few fields to see if the same point has
been reached before. The standard method for handling such situations is
2020-10-13 06:29:06 +02:00
to compute an array of the already-visited values. For example, consider again
2017-10-09 03:44:17 +02:00
the following query that searches a table <structname>graph</structname> using a
<structfield>link</structfield> field:
2008-10-13 18:25:20 +02:00
<programlisting>
WITH RECURSIVE search_graph(id, link, data, depth) AS (
2020-10-13 06:29:06 +02:00
SELECT g.id, g.link, g.data, 0
2018-04-05 02:57:26 +02:00
FROM graph g
UNION ALL
SELECT g.id, g.link, g.data, sg.depth + 1
FROM graph g, search_graph sg
WHERE g.id = sg.link
2008-10-13 18:25:20 +02:00
)
SELECT * FROM search_graph;
</programlisting>
2017-10-09 03:44:17 +02:00
This query will loop if the <structfield>link</structfield> relationships contain
cycles. Because we require a <quote>depth</quote> output, just changing
<literal>UNION ALL</literal> to <literal>UNION</literal> would not eliminate the looping.
2008-10-13 18:25:20 +02:00
Instead we need to recognize whether we have reached the same row again
while following a particular path of links. We add two columns
2020-10-12 07:46:20 +02:00
<structfield>is_cycle</structfield> and <structfield>path</structfield> to the loop-prone query:
2008-10-13 18:25:20 +02:00
<programlisting>
2020-10-13 06:29:06 +02:00
WITH RECURSIVE search_graph(id, link, data, depth, <emphasis>is_cycle, path</emphasis>) AS (
SELECT g.id, g.link, g.data, 0,
<emphasis>false,
ARRAY[g.id]</emphasis>
2018-04-05 02:57:26 +02:00
FROM graph g
UNION ALL
SELECT g.id, g.link, g.data, sg.depth + 1,
2020-10-13 06:29:06 +02:00
<emphasis>g.id = ANY(path),
path || g.id</emphasis>
2018-04-05 02:57:26 +02:00
FROM graph g, search_graph sg
2020-10-13 06:29:06 +02:00
WHERE g.id = sg.link <emphasis>AND NOT is_cycle</emphasis>
2008-10-13 18:25:20 +02:00
)
SELECT * FROM search_graph;
</programlisting>
Aside from preventing cycles, the array value is often useful in its own
2017-10-09 03:44:17 +02:00
right as representing the <quote>path</quote> taken to reach any particular row.
2008-10-13 18:25:20 +02:00
</para>
<para>
In the general case where more than one field needs to be checked to
recognize a cycle, use an array of rows. For example, if we needed to
2017-10-09 03:44:17 +02:00
compare fields <structfield>f1</structfield> and <structfield>f2</structfield>:
2008-10-13 18:25:20 +02:00
<programlisting>
2020-10-13 06:29:06 +02:00
WITH RECURSIVE search_graph(id, link, data, depth, <emphasis>is_cycle, path</emphasis>) AS (
SELECT g.id, g.link, g.data, 0,
<emphasis>false,
ARRAY[ROW(g.f1, g.f2)]</emphasis>
2018-04-05 02:57:26 +02:00
FROM graph g
UNION ALL
SELECT g.id, g.link, g.data, sg.depth + 1,
2020-10-13 06:29:06 +02:00
<emphasis>ROW(g.f1, g.f2) = ANY(path),
path || ROW(g.f1, g.f2)</emphasis>
2018-04-05 02:57:26 +02:00
FROM graph g, search_graph sg
2020-10-13 06:29:06 +02:00
WHERE g.id = sg.link <emphasis>AND NOT is_cycle</emphasis>
2008-10-13 18:25:20 +02:00
)
SELECT * FROM search_graph;
</programlisting>
</para>
<tip>
<para>
2017-10-09 03:44:17 +02:00
Omit the <literal>ROW()</literal> syntax in the common case where only one field
2008-10-13 18:25:20 +02:00
needs to be checked to recognize a cycle. This allows a simple array
rather than a composite-type array to be used, gaining efficiency.
</para>
</tip>
2021-02-01 13:54:59 +01:00
<para>
There is built-in syntax to simplify cycle detection. The above query can
also be written like this:
<programlisting>
WITH RECURSIVE search_graph(id, link, data, depth) AS (
SELECT g.id, g.link, g.data, 1
FROM graph g
UNION ALL
SELECT g.id, g.link, g.data, sg.depth + 1
FROM graph g, search_graph sg
WHERE g.id = sg.link
2021-02-27 08:11:14 +01:00
) <emphasis>CYCLE id SET is_cycle USING path</emphasis>
2021-02-01 13:54:59 +01:00
SELECT * FROM search_graph;
</programlisting>
and it will be internally rewritten to the above form. The
<literal>CYCLE</literal> clause specifies first the list of columns to
track for cycle detection, then a column name that will show whether a
2021-02-27 08:11:14 +01:00
cycle has been detected, and finally the name of another column that will track the
2021-02-01 13:54:59 +01:00
path. The cycle and path columns will implicitly be added to the output
rows of the CTE.
</para>
2008-10-14 02:41:35 +02:00
<tip>
<para>
2020-10-13 06:29:06 +02:00
The cycle path column is computed in the same way as the depth-first
2021-02-01 13:54:59 +01:00
ordering column show in the previous section. A query can have both a
<literal>SEARCH</literal> and a <literal>CYCLE</literal> clause, but a
depth-first search specification and a cycle detection specification would
create redundant computations, so it's more efficient to just use the
<literal>CYCLE</literal> clause and order by the path column. If
breadth-first ordering is wanted, then specifying both
<literal>SEARCH</literal> and <literal>CYCLE</literal> can be useful.
2008-10-14 02:41:35 +02:00
</para>
</tip>
2008-10-13 18:25:20 +02:00
<para>
A helpful trick for testing queries
2017-10-09 03:44:17 +02:00
when you are not certain if they might loop is to place a <literal>LIMIT</literal>
2008-10-07 21:27:04 +02:00
in the parent query. For example, this query would loop forever without
2017-10-09 03:44:17 +02:00
the <literal>LIMIT</literal>:
2008-10-04 23:56:55 +02:00
<programlisting>
WITH RECURSIVE t(n) AS (
SELECT 1
UNION ALL
SELECT n+1 FROM t
)
2020-10-13 06:29:06 +02:00
SELECT n FROM t <emphasis>LIMIT 100</emphasis>;
2008-10-04 23:56:55 +02:00
</programlisting>
This works because <productname>PostgreSQL</productname>'s implementation
2017-10-09 03:44:17 +02:00
evaluates only as many rows of a <literal>WITH</literal> query as are actually
2008-10-07 21:27:04 +02:00
fetched by the parent query. Using this trick in production is not
2008-10-14 02:41:35 +02:00
recommended, because other systems might work differently. Also, it
usually won't work if you make the outer query sort the recursive query's
2011-02-28 07:17:18 +01:00
results or join them to some other table, because in such cases the
2017-10-09 03:44:17 +02:00
outer query will usually try to fetch all of the <literal>WITH</literal> query's
2011-02-28 07:17:18 +01:00
output anyway.
2008-10-04 23:56:55 +02:00
</para>
2020-10-13 06:29:06 +02:00
</sect3>
</sect2>
<sect2>
<title>Common Table Expression Materialization</title>
2008-10-04 23:56:55 +02:00
<para>
Allow user control of CTE materialization, and change the default behavior.
Historically we've always materialized the full output of a CTE query,
treating WITH as an optimization fence (so that, for example, restrictions
from the outer query cannot be pushed into it). This is appropriate when
the CTE query is INSERT/UPDATE/DELETE, or is recursive; but when the CTE
query is non-recursive and side-effect-free, there's no hazard of changing
the query results by pushing restrictions down.
Another argument for materialization is that it can avoid duplicate
computation of an expensive WITH query --- but that only applies if
the WITH query is called more than once in the outer query. Even then
it could still be a net loss, if each call has restrictions that
would allow just a small part of the WITH query to be computed.
Hence, let's change the behavior for WITH queries that are non-recursive
and side-effect-free. By default, we will inline them into the outer
query (removing the optimization fence) if they are called just once.
If they are called more than once, we will keep the old behavior by
default, but the user can override this and force inlining by specifying
NOT MATERIALIZED. Lastly, the user can force the old behavior by
specifying MATERIALIZED; this would mainly be useful when the query had
deliberately been employing WITH as an optimization fence to prevent a
poor choice of plan.
Andreas Karlsson, Andrew Gierth, David Fetter
Discussion: https://postgr.es/m/87sh48ffhb.fsf@news-spur.riddles.org.uk
2019-02-16 22:11:12 +01:00
A useful property of <literal>WITH</literal> queries is that they are
normally evaluated only once per execution of the parent query, even if
they are referred to more than once by the parent query or
sibling <literal>WITH</literal> queries.
2008-10-04 23:56:55 +02:00
Thus, expensive calculations that are needed in multiple places can be
2017-10-09 03:44:17 +02:00
placed within a <literal>WITH</literal> query to avoid redundant work. Another
2008-10-04 23:56:55 +02:00
possible application is to prevent unwanted multiple evaluations of
functions with side-effects.
Allow user control of CTE materialization, and change the default behavior.
Historically we've always materialized the full output of a CTE query,
treating WITH as an optimization fence (so that, for example, restrictions
from the outer query cannot be pushed into it). This is appropriate when
the CTE query is INSERT/UPDATE/DELETE, or is recursive; but when the CTE
query is non-recursive and side-effect-free, there's no hazard of changing
the query results by pushing restrictions down.
Another argument for materialization is that it can avoid duplicate
computation of an expensive WITH query --- but that only applies if
the WITH query is called more than once in the outer query. Even then
it could still be a net loss, if each call has restrictions that
would allow just a small part of the WITH query to be computed.
Hence, let's change the behavior for WITH queries that are non-recursive
and side-effect-free. By default, we will inline them into the outer
query (removing the optimization fence) if they are called just once.
If they are called more than once, we will keep the old behavior by
default, but the user can override this and force inlining by specifying
NOT MATERIALIZED. Lastly, the user can force the old behavior by
specifying MATERIALIZED; this would mainly be useful when the query had
deliberately been employing WITH as an optimization fence to prevent a
poor choice of plan.
Andreas Karlsson, Andrew Gierth, David Fetter
Discussion: https://postgr.es/m/87sh48ffhb.fsf@news-spur.riddles.org.uk
2019-02-16 22:11:12 +01:00
However, the other side of this coin is that the optimizer is not able to
push restrictions from the parent query down into a multiply-referenced
<literal>WITH</literal> query, since that might affect all uses of the
<literal>WITH</literal> query's output when it should affect only one.
The multiply-referenced <literal>WITH</literal> query will be
2010-10-16 01:53:59 +02:00
evaluated as written, without suppression of rows that the parent query
2008-10-04 23:56:55 +02:00
might discard afterwards. (But, as mentioned above, evaluation might stop
early if the reference(s) to the query demand only a limited number of
rows.)
</para>
Allow user control of CTE materialization, and change the default behavior.
Historically we've always materialized the full output of a CTE query,
treating WITH as an optimization fence (so that, for example, restrictions
from the outer query cannot be pushed into it). This is appropriate when
the CTE query is INSERT/UPDATE/DELETE, or is recursive; but when the CTE
query is non-recursive and side-effect-free, there's no hazard of changing
the query results by pushing restrictions down.
Another argument for materialization is that it can avoid duplicate
computation of an expensive WITH query --- but that only applies if
the WITH query is called more than once in the outer query. Even then
it could still be a net loss, if each call has restrictions that
would allow just a small part of the WITH query to be computed.
Hence, let's change the behavior for WITH queries that are non-recursive
and side-effect-free. By default, we will inline them into the outer
query (removing the optimization fence) if they are called just once.
If they are called more than once, we will keep the old behavior by
default, but the user can override this and force inlining by specifying
NOT MATERIALIZED. Lastly, the user can force the old behavior by
specifying MATERIALIZED; this would mainly be useful when the query had
deliberately been employing WITH as an optimization fence to prevent a
poor choice of plan.
Andreas Karlsson, Andrew Gierth, David Fetter
Discussion: https://postgr.es/m/87sh48ffhb.fsf@news-spur.riddles.org.uk
2019-02-16 22:11:12 +01:00
<para>
However, if a <literal>WITH</literal> query is non-recursive and
side-effect-free (that is, it is a <literal>SELECT</literal> containing
no volatile functions) then it can be folded into the parent query,
allowing joint optimization of the two query levels. By default, this
happens if the parent query references the <literal>WITH</literal> query
just once, but not if it references the <literal>WITH</literal> query
more than once. You can override that decision by
specifying <literal>MATERIALIZED</literal> to force separate calculation
of the <literal>WITH</literal> query, or by specifying <literal>NOT
MATERIALIZED</literal> to force it to be merged into the parent query.
The latter choice risks duplicate computation of
the <literal>WITH</literal> query, but it can still give a net savings if
each usage of the <literal>WITH</literal> query needs only a small part
of the <literal>WITH</literal> query's full output.
</para>
<para>
A simple example of these rules is
<programlisting>
WITH w AS (
SELECT * FROM big_table
)
SELECT * FROM w WHERE key = 123;
</programlisting>
This <literal>WITH</literal> query will be folded, producing the same
execution plan as
<programlisting>
SELECT * FROM big_table WHERE key = 123;
</programlisting>
In particular, if there's an index on <structfield>key</structfield>,
it will probably be used to fetch just the rows having <literal>key =
123</literal>. On the other hand, in
<programlisting>
WITH w AS (
SELECT * FROM big_table
)
SELECT * FROM w AS w1 JOIN w AS w2 ON w1.key = w2.ref
WHERE w2.key = 123;
</programlisting>
the <literal>WITH</literal> query will be materialized, producing a
temporary copy of <structname>big_table</structname> that is then
joined with itself — without benefit of any index. This query
will be executed much more efficiently if written as
<programlisting>
WITH w AS NOT MATERIALIZED (
SELECT * FROM big_table
)
SELECT * FROM w AS w1 JOIN w AS w2 ON w1.key = w2.ref
WHERE w2.key = 123;
</programlisting>
so that the parent query's restrictions can be applied directly
to scans of <structname>big_table</structname>.
</para>
<para>
An example where <literal>NOT MATERIALIZED</literal> could be
undesirable is
<programlisting>
WITH w AS (
SELECT key, very_expensive_function(val) as f FROM some_table
)
SELECT * FROM w AS w1 JOIN w AS w2 ON w1.f = w2.f;
</programlisting>
Here, materialization of the <literal>WITH</literal> query ensures
that <function>very_expensive_function</function> is evaluated only
once per table row, not twice.
</para>
2010-10-16 01:53:59 +02:00
<para>
2017-10-09 03:44:17 +02:00
The examples above only show <literal>WITH</literal> being used with
<command>SELECT</command>, but it can be attached in the same way to
<command>INSERT</command>, <command>UPDATE</command>, or <command>DELETE</command>.
2010-10-16 01:53:59 +02:00
In each case it effectively provides temporary table(s) that can
be referred to in the main command.
</para>
2011-02-28 07:17:18 +01:00
</sect2>
<sect2 id="queries-with-modifying">
2017-10-09 03:44:17 +02:00
<title>Data-Modifying Statements in <literal>WITH</literal></title>
2011-02-28 07:17:18 +01:00
<para>
2017-10-09 03:44:17 +02:00
You can use data-modifying statements (<command>INSERT</command>,
<command>UPDATE</command>, or <command>DELETE</command>) in <literal>WITH</literal>. This
2011-02-28 07:17:18 +01:00
allows you to perform several different operations in the same query.
An example is:
<programlisting>
WITH moved_rows AS (
DELETE FROM products
WHERE
"date" >= '2010-10-01' AND
"date" < '2010-11-01'
RETURNING *
)
INSERT INTO products_log
SELECT * FROM moved_rows;
</programlisting>
2017-10-09 03:44:17 +02:00
This query effectively moves rows from <structname>products</structname> to
<structname>products_log</structname>. The <command>DELETE</command> in <literal>WITH</literal>
deletes the specified rows from <structname>products</structname>, returning their
contents by means of its <literal>RETURNING</literal> clause; and then the
2011-02-28 07:17:18 +01:00
primary query reads that output and inserts it into
2017-10-09 03:44:17 +02:00
<structname>products_log</structname>.
2011-02-28 07:17:18 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
A fine point of the above example is that the <literal>WITH</literal> clause is
attached to the <command>INSERT</command>, not the sub-<command>SELECT</command> within
the <command>INSERT</command>. This is necessary because data-modifying
statements are only allowed in <literal>WITH</literal> clauses that are attached
to the top-level statement. However, normal <literal>WITH</literal> visibility
rules apply, so it is possible to refer to the <literal>WITH</literal>
statement's output from the sub-<command>SELECT</command>.
2011-02-28 07:17:18 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
Data-modifying statements in <literal>WITH</literal> usually have
2017-11-23 15:39:47 +01:00
<literal>RETURNING</literal> clauses (see <xref linkend="dml-returning"/>),
2016-11-22 20:02:52 +01:00
as shown in the example above.
2017-10-09 03:44:17 +02:00
It is the output of the <literal>RETURNING</literal> clause, <emphasis>not</emphasis> the
2011-02-28 07:17:18 +01:00
target table of the data-modifying statement, that forms the temporary
table that can be referred to by the rest of the query. If a
2017-10-09 03:44:17 +02:00
data-modifying statement in <literal>WITH</literal> lacks a <literal>RETURNING</literal>
2011-02-28 07:17:18 +01:00
clause, then it forms no temporary table and cannot be referred to in
the rest of the query. Such a statement will be executed nonetheless.
A not-particularly-useful example is:
<programlisting>
WITH t AS (
DELETE FROM foo
)
DELETE FROM bar;
</programlisting>
2017-10-09 03:44:17 +02:00
This example would remove all rows from tables <structname>foo</structname> and
<structname>bar</structname>. The number of affected rows reported to the client
would only include rows removed from <structname>bar</structname>.
2011-02-28 07:17:18 +01:00
</para>
<para>
Recursive self-references in data-modifying statements are not
allowed. In some cases it is possible to work around this limitation by
2017-10-09 03:44:17 +02:00
referring to the output of a recursive <literal>WITH</literal>, for example:
2011-02-28 07:17:18 +01:00
<programlisting>
WITH RECURSIVE included_parts(sub_part, part) AS (
SELECT sub_part, part FROM parts WHERE part = 'our_product'
UNION ALL
SELECT p.sub_part, p.part
FROM included_parts pr, parts p
WHERE p.part = pr.sub_part
2018-04-05 02:57:26 +02:00
)
2011-02-28 07:17:18 +01:00
DELETE FROM parts
WHERE part IN (SELECT part FROM included_parts);
</programlisting>
This query would remove all direct and indirect subparts of a product.
</para>
<para>
2017-10-09 03:44:17 +02:00
Data-modifying statements in <literal>WITH</literal> are executed exactly once,
2011-02-28 07:17:18 +01:00
and always to completion, independently of whether the primary query
reads all (or indeed any) of their output. Notice that this is different
2017-10-09 03:44:17 +02:00
from the rule for <command>SELECT</command> in <literal>WITH</literal>: as stated in the
previous section, execution of a <command>SELECT</command> is carried only as far
2011-02-28 07:17:18 +01:00
as the primary query demands its output.
</para>
<para>
2017-10-09 03:44:17 +02:00
The sub-statements in <literal>WITH</literal> are executed concurrently with
2011-02-28 07:17:18 +01:00
each other and with the main query. Therefore, when using data-modifying
2017-10-09 03:44:17 +02:00
statements in <literal>WITH</literal>, the order in which the specified updates
2011-02-28 07:17:18 +01:00
actually happen is unpredictable. All the statements are executed with
2017-11-23 15:39:47 +01:00
the same <firstterm>snapshot</firstterm> (see <xref linkend="mvcc"/>), so they
2017-10-09 03:44:17 +02:00
cannot <quote>see</quote> one another's effects on the target tables. This
2011-02-28 07:17:18 +01:00
alleviates the effects of the unpredictability of the actual order of row
2017-10-09 03:44:17 +02:00
updates, and means that <literal>RETURNING</literal> data is the only way to
communicate changes between different <literal>WITH</literal> sub-statements and
2011-02-28 07:17:18 +01:00
the main query. An example of this is that in
<programlisting>
WITH t AS (
UPDATE products SET price = price * 1.05
RETURNING *
)
SELECT * FROM products;
</programlisting>
2017-10-09 03:44:17 +02:00
the outer <command>SELECT</command> would return the original prices before the
action of the <command>UPDATE</command>, while in
2011-02-28 07:17:18 +01:00
<programlisting>
WITH t AS (
UPDATE products SET price = price * 1.05
RETURNING *
)
SELECT * FROM t;
</programlisting>
2017-10-09 03:44:17 +02:00
the outer <command>SELECT</command> would return the updated data.
2011-02-28 07:17:18 +01:00
</para>
<para>
Trying to update the same row twice in a single statement is not
supported. Only one of the modifications takes place, but it is not easy
(and sometimes not possible) to reliably predict which one. This also
applies to deleting a row that was already updated in the same statement:
only the update is performed. Therefore you should generally avoid trying
to modify a single row twice in a single statement. In particular avoid
2017-10-09 03:44:17 +02:00
writing <literal>WITH</literal> sub-statements that could affect the same rows
2011-02-28 07:17:18 +01:00
changed by the main statement or a sibling sub-statement. The effects
of such a statement will not be predictable.
</para>
<para>
At present, any table used as the target of a data-modifying statement in
2017-10-09 03:44:17 +02:00
<literal>WITH</literal> must not have a conditional rule, nor an <literal>ALSO</literal>
rule, nor an <literal>INSTEAD</literal> rule that expands to multiple statements.
2011-02-28 07:17:18 +01:00
</para>
</sect2>
2010-10-16 01:53:59 +02:00
2008-10-04 23:56:55 +02:00
</sect1>
2001-01-23 00:34:33 +01:00
</chapter>