Queries
A query is the process of or the command to
retrieve data from a database. In SQL the SELECT
command is used to specify queries. The general syntax of the
SELECT command is
SELECT select_list FROM table_expression sort_specification
The following sections describe the details of the select list, the
table expression, and the sort specification. The simplest kind of
query has the form
SELECT * FROM table1;
Assuming that there is a table called table1, this command would
retrieve all rows and all columns from table1. (The method of
retrieval depends on the client application. For example, the
psql program will display an ASCII-art
table on the screen, client libraries will offer functions to
retrieve individual rows and columns.) The select list
specification * means all columns that the table
expression happens to provide. A select list can also select a
subset of the available columns or even make calculations on the
columns before retrieving them; see . For example, if table1 has columns
named a, b, and c (and perhaps others) you can make the following
query:
SELECT a, b + c FROM table1;
(assuming that b and c are of a numeric data type).
FROM table1 is a particularly simple kind of
table expression. In general, table expressions can be complex
constructs of base tables, joins, and subqueries. But you can also
omit the table expression entirely and use the SELECT command as a
calculator:
SELECT 3 * 4;
This is more useful if the expressions in the select list return
varying results. For example, you could call a function this way.
SELECT random();
Table Expressions
A table expression specifies a table. The
table expression contains a FROM clause that is optionally followed
by WHERE, GROUP BY, and HAVING clauses. Trivial table expressions
simply refer to a table on disk, a so-called base table, but more
complex expressions can be used to modify or combine base tables in
various ways.
The WHERE, GROUP BY, and HAVING clauses in the table expression
specify a pipeline of successive transformations performed on the
table derived in the FROM clause. The final transformed table that
is derived provides the input rows used to derive output rows as
specified by the select list of derived column value expressions.
FROM clause
The FROM clause derives a table from one or more other tables
given in a comma-separated table reference list.
FROM table_reference , table_reference , ...
A table reference may be a table name or a derived table such as a
subquery, a table join, or complex combinations of these. If more
than one table reference is listed in the FROM clause they are
CROSS JOINed (see below) to form the derived table that may then
be subject to transformations by the WHERE, GROUP BY, and HAVING
clauses and is finally the result of the overall table expression.
If a table reference is a simple table name and it is the
supertable in a table inheritance hierarchy, rows of the table
include rows from all of its subtable successors unless the
keyword ONLY precedes the table name.
Joined Tables
A joined table is a table derived from two other (real or
derived) tables according to the rules of the particular join
type. INNER, OUTER, NATURAL, and CROSS JOIN are supported.
Join Types
CROSS JOIN
T1 CROSS JOIN T2
For each combination of rows from
T1 and
T2 the derived table will contain a
row consisting of all columns in T1
followed by all columns in T2. If
the tables have have N and M rows respectively, the joined
table will have N * M rows. A cross join is essentially an
INNER JOIN ON TRUE.
FROM T1 CROSS JOIN
T2 is equivalent to
FROM T1,
T2.
Qualified JOINs
T1 { INNER | { LEFT | RIGHT | FULL } OUTER } JOIN T2 ON boolean expression
T1 { INNER | { LEFT | RIGHT | FULL } OUTER } JOIN T2 USING ( join column list )
The words INNER and OUTER are
optional for all JOINs. INNER is the default;
LEFT, RIGHT, and
FULL are for OUTER JOINs only.
The join condition is specified in the
ON or USING clause. (The meaning of the join condition
depends on the particular join type; see below.) The ON
clause takes a Boolean value expression of the same kind as is
used in a WHERE clause. The USING clause takes a
comma-separated list of column names, which the joined tables
must have in common, and joins the tables on the equality of
those columns as a set, resulting in a joined table having one
column for each common column listed and all of the other
columns from both tables. Thus, USING (a, b,
c) is equivalent to ON (t1.a = t2.a AND
t1.b = t2.b AND t1.c = t2.c) with the exception that
if ON is used there will be two columns a, b, and c in the
result, whereas with USING there will be only one of each.
INNER JOIN
For each row R1 of T1, the joined table has a row for each
row in T2 that satisfies the join condition with R1.
LEFT OUTER JOIN
First, an INNER JOIN is performed. Then, for a row in T1
that does not satisfy the join condition with any row in
T2, a joined row is returned with NULL values in columns of
T2. Thus, the joined table unconditionally has a row for each
row in T1.
RIGHT OUTER JOIN
This is like a left join, only that the result table will
unconditionally have a row for each row in T2.
FULL OUTER JOIN
First, an INNER JOIN is performed. Then, for each row in
T1 that does not satisfy the join condition with any row in
T2, a joined row is returned with null values in columns of
T2. Also, for each row of T2 that does not satisfy the
join condition with any row in T1, a joined row with null
values in the columns of T1 is returned.
NATURAL JOIN
T1 NATURAL { INNER | { LEFT | RIGHT | FULL } OUTER JOIN T2
A natural join creates a joined table where every pair of matching
column names between the two tables are merged into one column. The
join specification is effectively a USING clause containing all the
common column names and is otherwise like a Qualified JOIN.
Joins of all types can be chained together or nested where either
or both of T1 and
T2 may be JOINed tables. Parenthesis
can be used around JOIN clauses to control the join order which
are otherwise left to right.
Subqueries
Subqueries specifying a derived table must be enclosed in
parenthesis and must be named using an AS
clause. (See .)
FROM (SELECT * FROM table1) AS alias_name
This example is equivalent to FROM table1 AS
alias_name. Many subqueries can be written as table
joins instead.
Table and Column Aliases
A temporary name can be given to tables and complex table
references to be used for references to the derived table in
further processing. This is called a table
alias.
FROM table_reference AS alias
Here, alias can be any regular
identifier. The alias becomes the new name of the table
reference for the current query -- it is no longer possible to
refer to the table by the original name (if the table reference
was an ordinary base table). Thus
SELECT * FROM my_table AS m WHERE my_table.a > 5;
is not valid SQL syntax. What will happen instead, as a
Postgres extension, is that an implicit
table reference is added to the FROM clause, so the query is
processed as if it was written as
SELECT * FROM my_table AS m, my_table WHERE my_table.a > 5;
Table aliases are mainly for notational convenience, but it is
necessary to use them when joining a table to itself, e.g.,
SELECT * FROM my_table AS a CROSS JOIN my_table AS b ...
Additionally, an alias is required if the table reference is a
subquery.
Parenthesis are used to resolve ambiguities. The following
statement will assign the alias b to the
result of the join, unlike the previous example:
SELECT * FROM (my_table AS a CROSS JOIN my_table) AS b ...
FROM table_reference alias
This form is equivalent the previously treated one; the
AS key word is noise.
FROM table_reference AS alias ( column1 , column2 , ... )
In addition to renaming the table as described above, the columns
of the table are also given temporary names. If less column
aliases are specified than the actual table has columns, the last
columns are not renamed. This syntax is especially useful for
self-joins or subqueries.
Examples
FROM T1 INNER JOIN T2 USING (C)
FROM T1 LEFT OUTER JOIN T2 USING (C)
FROM (T1 RIGHT OUTER JOIN T2 ON (T1C1=T2C1)) AS DT1
FROM (T1 FULL OUTER JOIN T2 USING (C)) AS DT1 (DT1C1, DT1C2)
FROM T1 NATURAL INNER JOIN T2
FROM T1 NATURAL LEFT OUTER JOIN T2
FROM T1 NATURAL RIGHT OUTER JOIN T2
FROM T1 NATURAL FULL OUTER JOIN T2
FROM (SELECT * FROM T1) DT1 CROSS JOIN T2, T3
FROM (SELECT * FROM T1) DT1, T2, T3
Above are some examples of joined tables and complex derived
tables. Notice how the AS clause renames or names a derived
table and how the optional comma-separated list of column names
that follows gives names or renames the columns. The last two
FROM clauses produce the same derived table from T1, T2, and T3.
The AS keyword was omitted in naming the subquery as DT1. The
keywords OUTER and INNER are noise that can be omitted also.
WHERE clause
The syntax of the WHERE clause is
WHERE search condition
where search condition is any value
expression as defined in that
returns a value of type boolean.
After the processing of the FROM clause is done, each row of the
derived table is checked against the search condition. If the
result of the condition is true, the row is kept in the output
table, otherwise (that is, if the result is false or NULL) it is
discarded. The search condition typically references at least some
column in the table generated in the FROM clause; this is not
required, but otherwise the WHERE clause will be fairly useless.
Before the implementation of the JOIN syntax, it was necessary to
put the join condition of an inner join in the WHERE clause. For
example, these table expressions are equivalent:
FROM a, b WHERE a.id = b.id AND b.val > 5
and
FROM a INNER JOIN b ON (a.id = b.id) WHERE b.val > 5
or perhaps even
FROM a NATURAL JOIN b WHERE b.val > 5
Which one of these you use is mainly a matter of style. The JOIN
syntax in the FROM clause is probably not as portable to other
products. For outer joins there is no choice in any case: they
must be done in the FROM clause.
FROM FDT WHERE
C1 > 5
FROM FDT WHERE
C1 IN (1, 2, 3)
FROM FDT WHERE
C1 IN (SELECT C1 FROM T2)
FROM FDT WHERE
C1 IN (SELECT C3 FROM T2 WHERE C2 = FDT.C1 + 10)
FROM FDT WHERE
C1 BETWEEN (SELECT C3 FROM T2 WHERE C2 = FDT.C1 + 10) AND 100
FROM FDT WHERE
EXISTS (SELECT C1 FROM T2 WHERE C2 > FDT.C1)
In the examples above, FDT is the table derived in the FROM
clause. Rows that do not meet the search condition of the where
clause are eliminated from FDT. Notice the use of scalar
subqueries as value expressions (C2 assumed UNIQUE). Just like
any other query, the subqueries can employ complex table
expressions. Notice how FDT is referenced in the subqueries.
Qualifying C1 as FDT.C1 is only necessary if C1 is the name of a
column in the derived input table of the subquery. Qualifying the
column name adds clarity even when it is not needed. The column
naming scope of an outer query extends into its inner queries.
GROUP BY and HAVING clauses
After passing the WHERE filter, the derived input table may be
subject to grouping, using the GROUP BY clause, and elimination of
group rows using the HAVING clause.
SELECT select_list FROM ... WHERE ... GROUP BY grouping_column_reference , grouping_column_reference...
The GROUP BY clause is used to group together rows in a table that
share the same values in all the columns listed. The order in
which the columns are listed does not matter (as opposed to an
ORDER BY clause). The purpose is to reduce each group of rows
sharing common values into one group row that is representative of
all rows in the group. This is done to eliminate redundancy in
the output and/or obtain aggregates that apply to these groups.
Once a table is grouped, columns that are not included in the
grouping cannot be referenced, except in aggregate expressions,
since a specific value in those columns is ambiguous - which row
in the group should it come from? The grouped-by columns can be
referenced in select list column expressions since they have a
known constant value per group. Aggregate functions on the
ungrouped columns provide values that span the rows of a group,
not of the whole table. For instance, a
sum(sales) on a grouped table by product code
gives the total sales for each product, not the total sales on all
products. The aggregates of the ungrouped columns are
representative of the group, whereas their individual values may
not be.
Example:
SELECT pid, p.name, (sum(s.units) * p.price) AS sales
FROM products p LEFT JOIN sales s USING ( pid )
GROUP BY pid, p.name, p.price;
In this example, the columns pid, p.name, and p.price must be in
the GROUP BY clause since they are referenced in the query select
list. The column s.units does not have to be in the GROUP BY list
since it is only used in an aggregate expression
(sum()), which represents the group of sales
of a product. For each product, a summary row is returned about
all sales of the product.
In strict SQL, GROUP BY can only group by columns of the source
table but Postgres extends this to also allow GROUP BY to group by
select columns in the query select list. Grouping by value
expressions instead of simple column names is also allowed.
SELECT select_list FROM ... WHERE ... GROUP BY ... HAVING boolean_expression
If a table has been grouped using a GROUP BY clause, but then only
certain groups are of interest, the HAVING clause can be used,
much like a WHERE clause, to eliminate groups from a grouped
table. For some queries, Postgres allows a HAVING clause to be
used without a GROUP BY and then it acts just like another WHERE
clause, but the point in using HAVING that way is not clear. Since
HAVING operates on groups, only grouped columns can be listed in
the HAVING clause. If selection based on some ungrouped column is
desired, it should be expressed in the WHERE clause.
Example:
SELECT pid AS "Products",
p.name AS "Over 5000",
(sum(s.units) * (p.price - p.cost)) AS "Past Month Profit"
FROM products p LEFT JOIN sales s USING ( pid )
WHERE p.date > CURRENT_DATE - INTERVAL '4 weeks'
GROUP BY pid, p.name, p.price, p.cost
HAVING p.price > 5000;
In the example above, the WHERE clause is selecting rows by a
column that is not grouped, while the HAVING clause
is selecting groups with a price greater than 5000.
Select Lists
The table expression in the SELECT command
constructs an intermediate virtual table by possibly combining
tables, views, eliminating rows, grouping, etc. This table is
finally passed on to processing by the select list. The select
list determines which columns of the
intermediate table are retained. The simplest kind of select list
is * which retains all columns that the table
expression produces. Otherwise, a select list is a comma-separated
list of value expressions (as defined in ). For instance, it could be a list of
column names:
SELECT a, b, c FROM ...
The columns names a, b, and c are either the actual names of the
columns of table referenced in the FROM clause, or the aliases
given to them as explained in .
The name space available in the select list is the same as in the
WHERE clause (unless grouping is used, in which case it is the same
as in the HAVING clause). If more than one table has a column of
the same name, the table name must also be given, as in
SELECT tbl1.a, tbl2.b, tbl1.c FROM ...
(see also ).
If an arbitrary value expression is used in the select list, it
conceptually adds a new virtual column to the returned table. The
value expression is effectively evaluated once for each retrieved
row with real values substituted for any column references. But
the expressions in the select list do not have to reference any
columns in the table expression of the FROM clause; they can be
constant arithmetic expressions as well, for instance.
Column Labels
The entries in the select list can be assigned names for further
processing. The further processing
in this case is
an optional sort specification and the client application (e.g.,
column headers for display). For example:
SELECT a AS value, b + c AS sum FROM ...
The AS key word can in fact be omitted.
If no name is chosen, the system assigns a default. For simple
column references, this is the name of the column. For function
calls, this is the name of the function. For complex expressions,
the system will generate a generic name.
The naming of output columns here is different from that done in
the FROM clause (see ). This
pipeline will in fact allow you to rename the same column twice,
but the name chosen in the select list is the one that will be
passed on.
DISTINCT
After the select list has been processed, the result table may
optionally be subject to the elimination of duplicates. The
DISTINCT key word is written directly after the
SELECT to enable this:
SELECT DISTINCT select_list ...
(Instead of DISTINCT the word ALL
can be used to select the default behavior of retaining all rows.)
Obviously, two rows are considered distinct if they differ in at
least one column value. NULLs are considered equal in this
consideration.
Alternatively, an arbitrary expression can determine what rows are
to be considered distinct:
SELECT DISTINCT ON (expression , expression ...) select_list ...
Here expression is an arbitrary value
expression that is evaluated for all rows. A set of rows for
which all the expressions is equal are considered duplicates and
only the first row is kept in the output. Note that the
first row
of a set is unpredictable unless the
query is sorted.
The DISTINCT ON clause is not part of the SQL standard and is
sometimes considered bad style because of the indeterminate nature
of its results. With judicious use of GROUP BY and subselects in
FROM the construct can be avoided, but it is very often the much
more convenient alternative.
Combining Queries
The results of two queries can be combined using the set operations
union, intersection, and difference. The syntax is
query1 UNION ALL query2
query1 INTERSECT ALL query2
query1 EXCEPT ALL query2
query1 and
query2 are queries that can use any of
the features discussed up to this point. Set operations can also
be nested and chained, for example
query1 UNION query2 UNION query3
which really says
(query1 UNION query2) UNION query3
UNION effectively appends the result of
query2 to the result of
query1 (although there is no guarantee
that this is the order in which the rows are actually returned) and
eliminates all duplicate rows, in the sense of DISTINCT, unless ALL
is specified.
INTERSECT returns all rows that are both in the
result of query1 and in the result of
query2. Duplicate rows are eliminated
unless ALL is specified.
EXCEPT returns all rows that are in the result
of query1 but not in the result of
query2. Again, duplicates are
eliminated unless ALL is specified.
In order to calculate the union, intersection, or difference of two
queries, the two queries must be union compatible
,
which means that they both return the same number of columns, and
that the corresponding columns have compatible data types, as
described in .
Sorting Rows
After a query has produced an output table (after the select list
has been processed) it can optionally be sorted. If sorting is not
chosen, the rows will be returned in random order. The actual
order in that case will depend on the scan and join plan types and
the order on disk, but it must not be relied on. A particular
ordering can only be guaranteed if the sort step is explicitly
chosen.
The ORDER BY clause specifies the sort order:
SELECT select_list FROM table_expression ORDER BY column1 ASC | DESC , column2 ASC | DESC ...
column1, etc., refer to select list
columns: It can either be the name of a column (either the
explicit column label or default name, as explained in ) or the number of a column. Some
examples:
SELECT a, b FROM table1 ORDER BY a;
SELECT a + b AS sum, c FROM table1 ORDER BY sum;
SELECT a, sum(b) FROM table1 GROUP BY a ORDER BY 1;
As an extension to the SQL standard, Postgres also allows ordering
by arbitrary expressions:
SELECT a, b FROM table1 ORDER BY a + b;
References to column names in the FROM clause that are renamed in
the select list are also allowed:
SELECT a AS b FROM table1 ORDER BY a;
But this does not work in queries involving UNION, INTERSECT, or
EXCEPT, and is not portable.
Each column specification may be followed by an optional ASC or
DESC to set the sort direction. ASC is default. Ascending order
puts smaller values first, where smaller
is defined
in terms of the < operator. Similarly,
descending order is determined with the >
operator.
If more than one sort column is specified the later entries are
used to sort the rows that are equal under the order imposed by the
earlier sort specifications.
LIMIT and OFFSET
SELECT select_list FROM table_expression ORDER BY sort_spec LIMIT { number | ALL } OFFSET number
LIMIT allows you to retrieve just a portion of the rows that are
generated by the rest of the query. If a limit count is given, no
more than that many rows will be returned. If an offset is given,
that many rows will be skipped before starting to return rows.
When using LIMIT, it is a good idea to use an ORDER BY clause that
constrains the result rows into a unique order. Otherwise you will
get an unpredictable subset of the query's rows---you may be asking
for the tenth through twentieth rows, but tenth through twentieth
in what ordering? The ordering is unknown, unless you specified
ORDER BY.
The query optimizer takes LIMIT into account when generating a
query plan, so you are very likely to get different plans (yielding
different row orders) depending on what you give for LIMIT and
OFFSET. Thus, using different LIMIT/OFFSET values to select
different subsets of a query result will give
inconsistent results unless you enforce a predictable
result ordering with ORDER BY. This is not a bug; it is an
inherent consequence of the fact that SQL does not promise to
deliver the results of a query in any particular order unless ORDER
BY is used to constrain the order.