2010-09-20 22:08:53 +02:00
<!-- doc/src/sgml/extend.sgml -->
2000-03-31 05:27:42 +02:00
1999-07-22 17:11:05 +02:00
<chapter id="extend">
2003-04-10 03:22:45 +02:00
<title>Extending <acronym>SQL</acronym></title>
1999-07-22 17:11:05 +02:00
2007-08-31 23:33:48 +02:00
<indexterm zone="extend">
<primary>extending SQL</primary>
</indexterm>
2002-11-03 02:31:32 +01:00
1999-07-22 17:11:05 +02:00
<para>
In the sections that follow, we will discuss how you
2010-11-23 21:27:50 +01:00
can extend the <productname>PostgreSQL</productname>
1999-07-22 17:11:05 +02:00
<acronym>SQL</acronym> query language by adding:
<itemizedlist spacing="compact" mark="bullet">
<listitem>
<para>
2017-11-23 15:39:47 +01:00
functions (starting in <xref linkend="xfunc"/>)
1999-07-22 17:11:05 +02:00
</para>
</listitem>
2003-08-10 00:50:22 +02:00
<listitem>
<para>
2017-11-23 15:39:47 +01:00
aggregates (starting in <xref linkend="xaggr"/>)
2003-08-10 00:50:22 +02:00
</para>
</listitem>
1999-07-22 17:11:05 +02:00
<listitem>
<para>
2017-11-23 15:39:47 +01:00
data types (starting in <xref linkend="xtypes"/>)
1999-07-22 17:11:05 +02:00
</para>
</listitem>
<listitem>
<para>
2017-11-23 15:39:47 +01:00
operators (starting in <xref linkend="xoper"/>)
1999-07-22 17:11:05 +02:00
</para>
</listitem>
<listitem>
<para>
2017-11-23 15:39:47 +01:00
operator classes for indexes (starting in <xref linkend="xindex"/>)
1999-07-22 17:11:05 +02:00
</para>
</listitem>
2011-02-08 22:08:41 +01:00
<listitem>
<para>
2017-11-23 15:39:47 +01:00
packages of related objects (starting in <xref linkend="extend-extensions"/>)
2011-02-08 22:08:41 +01:00
</para>
</listitem>
1999-07-22 17:11:05 +02:00
</itemizedlist>
</para>
2000-09-29 22:21:34 +02:00
<sect1 id="extend-how">
1999-07-22 17:11:05 +02:00
<title>How Extensibility Works</title>
<para>
2010-11-23 21:27:50 +01:00
<productname>PostgreSQL</productname> is extensible because its operation is
catalog-driven. If you are familiar with standard
2003-04-10 03:22:45 +02:00
relational database systems, you know that they store information
1999-07-22 17:11:05 +02:00
about databases, tables, columns, etc., in what are
commonly known as system catalogs. (Some systems call
2003-08-10 00:50:22 +02:00
this the data dictionary.) The catalogs appear to the
2001-01-14 00:58:55 +01:00
user as tables like any other, but the <acronym>DBMS</acronym> stores
1999-07-22 17:11:05 +02:00
its internal bookkeeping in them. One key difference
2003-04-10 03:22:45 +02:00
between <productname>PostgreSQL</productname> and standard relational database systems is
2010-11-23 21:27:50 +01:00
that <productname>PostgreSQL</productname> stores much more information in its
2003-04-10 03:22:45 +02:00
catalogs: not only information about tables and columns,
but also information about data types, functions, access
2001-01-14 00:58:55 +01:00
methods, and so on. These tables can be modified by
2010-11-23 21:27:50 +01:00
the user, and since <productname>PostgreSQL</productname> bases its operation
2001-11-21 06:53:41 +01:00
on these tables, this means that <productname>PostgreSQL</productname> can be
1999-07-22 17:11:05 +02:00
extended by users. By comparison, conventional
2010-11-23 21:27:50 +01:00
database systems can only be extended by changing hardcoded
2003-04-10 03:22:45 +02:00
procedures in the source code or by loading modules
2002-01-07 03:29:15 +01:00
specially written by the <acronym>DBMS</acronym> vendor.
1999-07-22 17:11:05 +02:00
</para>
<para>
2003-09-11 23:42:20 +02:00
The <productname>PostgreSQL</productname> server can moreover
incorporate user-written code into itself through dynamic loading.
That is, the user can specify an object code file (e.g., a shared
library) that implements a new type or function, and
<productname>PostgreSQL</productname> will load it as required.
Code written in <acronym>SQL</acronym> is even more trivial to add
to the server. This ability to modify its operation <quote>on the
fly</quote> makes <productname>PostgreSQL</productname> uniquely
suited for rapid prototyping of new applications and storage
structures.
1999-07-22 17:11:05 +02:00
</para>
</sect1>
2003-08-31 19:32:24 +02:00
<sect1 id="extend-type-system">
2001-11-21 06:53:41 +01:00
<title>The <productname>PostgreSQL</productname> Type System</title>
1999-07-22 17:11:05 +02:00
2003-08-31 19:32:24 +02:00
<indexterm zone="extend-type-system">
<primary>base type</primary>
</indexterm>
<indexterm zone="extend-type-system">
<primary>data type</primary>
<secondary>base</secondary>
</indexterm>
<indexterm zone="extend-type-system">
<primary>composite type</primary>
2002-11-03 02:31:32 +01:00
</indexterm>
2003-08-31 19:32:24 +02:00
<indexterm zone="extend-type-system">
<primary>data type</primary>
<secondary>composite</secondary>
2002-11-03 02:31:32 +01:00
</indexterm>
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
<indexterm zone="extend-type-system">
<primary>container type</primary>
</indexterm>
<indexterm zone="extend-type-system">
<primary>data type</primary>
<secondary>container</secondary>
</indexterm>
1999-07-22 17:11:05 +02:00
<para>
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
<productname>PostgreSQL</productname> data types can be divided into base
types, container types, domains, and pseudo-types.
2003-08-10 00:50:22 +02:00
</para>
2023-01-09 21:08:24 +01:00
<sect2 id="extend-type-system-base">
2003-11-12 23:47:47 +01:00
<title>Base Types</title>
<para>
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
Base types are those, like <type>integer</type>, that are
2017-10-09 03:44:17 +02:00
implemented below the level of the <acronym>SQL</acronym> language
2003-11-12 23:47:47 +01:00
(typically in a low-level language such as C). They generally
correspond to what are often known as abstract data types.
<productname>PostgreSQL</productname> can only operate on such
types through functions provided by the user and only understands
the behavior of such types to the extent that the user describes
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
them.
2017-11-23 15:39:47 +01:00
The built-in base types are described in <xref linkend="datatype"/>.
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
</para>
<para>
Enumerated (enum) types can be considered as a subcategory of base
types. The main difference is that they can be created using
just <acronym>SQL</acronym> commands, without any low-level programming.
2017-11-23 15:39:47 +01:00
Refer to <xref linkend="datatype-enum"/> for more information.
2003-11-12 23:47:47 +01:00
</para>
</sect2>
2023-01-09 21:08:24 +01:00
<sect2 id="extend-type-system-container">
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
<title>Container Types</title>
<para>
<productname>PostgreSQL</productname> has three kinds
of <quote>container</quote> types, which are types that contain multiple
values of other types. These are arrays, composites, and ranges.
</para>
<para>
Arrays can hold multiple values that are all of the same type. An array
type is automatically created for each base type, composite type, range
type, and domain type. But there are no arrays of arrays. So far as
the type system is concerned, multi-dimensional arrays are the same as
2017-11-23 15:39:47 +01:00
one-dimensional arrays. Refer to <xref linkend="arrays"/> for more
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
information.
</para>
2003-11-12 23:47:47 +01:00
<para>
Composite types, or row types, are created whenever the user
2005-10-15 03:47:12 +02:00
creates a table. It is also possible to use <xref
2017-11-23 15:39:47 +01:00
linkend="sql-createtype"/> to
2017-10-09 03:44:17 +02:00
define a <quote>stand-alone</quote> composite type with no associated
2005-10-15 03:47:12 +02:00
table. A composite type is simply a list of types with
associated field names. A value of a composite type is a row or
2017-11-23 15:39:47 +01:00
record of field values. Refer to <xref linkend="rowtypes"/>
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
for more information.
</para>
<para>
A range type can hold two values of the same type, which are the lower
and upper bounds of the range. Range types are user-created, although
2017-11-23 15:39:47 +01:00
a few built-in ones exist. Refer to <xref linkend="rangetypes"/>
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
for more information.
2003-11-12 23:47:47 +01:00
</para>
</sect2>
2013-05-08 03:23:21 +02:00
<sect2 id="extend-type-system-domains">
2003-11-12 23:47:47 +01:00
<title>Domains</title>
<para>
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
A domain is based on a particular underlying type and for many purposes
is interchangeable with its underlying type. However, a domain can have
constraints that restrict its valid values to a subset of what the
underlying type would allow. Domains are created using
2017-11-23 15:39:47 +01:00
the <acronym>SQL</acronym> command <xref linkend="sql-createdomain"/>.
Refer to <xref linkend="domains"/> for more information.
2003-11-12 23:47:47 +01:00
</para>
</sect2>
2023-01-09 21:08:24 +01:00
<sect2 id="extend-type-system-pseudo">
2003-11-12 23:47:47 +01:00
<title>Pseudo-Types</title>
<para>
2017-10-09 03:44:17 +02:00
There are a few <quote>pseudo-types</quote> for special purposes.
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
Pseudo-types cannot appear as columns of tables or components of
container types, but they can be used to declare the argument and
2003-11-12 23:47:47 +01:00
result types of functions. This provides a mechanism within the
type system to identify special classes of functions. <xref
2017-11-23 15:39:47 +01:00
linkend="datatype-pseudotypes-table"/> lists the existing
2003-11-12 23:47:47 +01:00
pseudo-types.
</para>
</sect2>
2003-08-10 00:50:22 +02:00
2003-08-31 19:32:24 +02:00
<sect2 id="extend-types-polymorphic">
2003-11-12 23:47:47 +01:00
<title>Polymorphic Types</title>
2003-08-10 00:50:22 +02:00
2003-08-31 19:32:24 +02:00
<indexterm zone="extend-types-polymorphic">
<primary>polymorphic type</primary>
</indexterm>
<indexterm zone="extend-types-polymorphic">
<primary>polymorphic function</primary>
</indexterm>
<indexterm zone="extend-types-polymorphic">
Documentation improvements around domain types.
I was a bit surprised to find that domains were almost completely
unmentioned in the main SGML documentation, outside of the reference
pages for CREATE/ALTER/DROP DOMAIN. In particular, noplace was it
mentioned that we don't support domains over composite, making it
hard to document the planned fix for that.
Hence, add a section about domains to chapter 8 (Data Types).
Also, modernize the type system overview in section 37.2; it had never
heard of range types, and insisted on calling arrays base types, which
seems a bit odd from a user's perspective; furthermore it didn't fit well
with the fact that we now support arrays over types other than base types.
It seems appropriate to use the term "container types" to describe all of
arrays, composites, and ranges, so let's do that.
Also a few other minor improvements, notably improve an example query
in rowtypes.sgml by using a LATERAL function instead of an ad-hoc
OFFSET 0 clause.
In part this is mop-up for commit c12d570fa, which missed updating 37.2
to reflect the fact that it added arrays of domains. We could possibly
back-patch this without that claim, but I don't feel a strong need to.
2017-10-24 20:08:40 +02:00
<primary>data type</primary>
2003-08-31 19:32:24 +02:00
<secondary>polymorphic</secondary>
2003-08-10 00:50:22 +02:00
</indexterm>
2003-08-31 19:32:24 +02:00
<indexterm zone="extend-types-polymorphic">
<primary>function</primary>
<secondary>polymorphic</secondary>
2003-08-10 00:50:22 +02:00
</indexterm>
<para>
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
Some pseudo-types of special interest are the <firstterm>polymorphic
types</firstterm>, which are used to declare <firstterm>polymorphic
functions</firstterm>. This powerful feature allows a single function
definition to operate on many different data types, with the specific
data type(s) being determined by the data types actually passed to it
in a particular call. The polymorphic types are shown in
<xref linkend="extend-types-polymorphic-table"/>. Some examples of
their use appear in <xref linkend="xfunc-sql-polymorphic-functions"/>.
2003-08-10 00:50:22 +02:00
</para>
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
<table id="extend-types-polymorphic-table">
<title>Polymorphic Types</title>
<tgroup cols="3">
2020-05-06 18:23:43 +02:00
<colspec colname="col1" colwidth="2*"/>
<colspec colname="col2" colwidth="1*"/>
<colspec colname="col3" colwidth="2*"/>
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
<thead>
<row>
<entry>Name</entry>
<entry>Family</entry>
<entry>Description</entry>
</row>
</thead>
<tbody>
<row>
<entry><type>anyelement</type></entry>
<entry>Simple</entry>
<entry>Indicates that a function accepts any data type</entry>
</row>
<row>
<entry><type>anyarray</type></entry>
<entry>Simple</entry>
<entry>Indicates that a function accepts any array data type</entry>
</row>
<row>
<entry><type>anynonarray</type></entry>
<entry>Simple</entry>
<entry>Indicates that a function accepts any non-array data type</entry>
</row>
<row>
<entry><type>anyenum</type></entry>
<entry>Simple</entry>
<entry>Indicates that a function accepts any enum data type
(see <xref linkend="datatype-enum"/>)
</entry>
</row>
<row>
<entry><type>anyrange</type></entry>
<entry>Simple</entry>
<entry>Indicates that a function accepts any range data type
(see <xref linkend="rangetypes"/>)
</entry>
</row>
Multirange datatypes
Multiranges are basically sorted arrays of non-overlapping ranges with
set-theoretic operations defined over them.
Since v14, each range type automatically gets a corresponding multirange
datatype. There are both manual and automatic mechanisms for naming multirange
types. Once can specify multirange type name using multirange_type_name
attribute in CREATE TYPE. Otherwise, a multirange type name is generated
automatically. If the range type name contains "range" then we change that to
"multirange". Otherwise, we add "_multirange" to the end.
Implementation of multiranges comes with a space-efficient internal
representation format, which evades extra paddings and duplicated storage of
oids. Altogether this format allows fetching a particular range by its index
in O(n).
Statistic gathering and selectivity estimation are implemented for multiranges.
For this purpose, stored multirange is approximated as union range without gaps.
This field will likely need improvements in the future.
Catversion is bumped.
Discussion: https://postgr.es/m/CALNJ-vSUpQ_Y%3DjXvTxt1VYFztaBSsWVXeF1y6gTYQ4bOiWDLgQ%40mail.gmail.com
Discussion: https://postgr.es/m/a0b8026459d1e6167933be2104a6174e7d40d0ab.camel%40j-davis.com#fe7218c83b08068bfffb0c5293eceda0
Author: Paul Jungwirth, revised by me
Reviewed-by: David Fetter, Corey Huinker, Jeff Davis, Pavel Stehule
Reviewed-by: Alvaro Herrera, Tom Lane, Isaac Morland, David G. Johnston
Reviewed-by: Zhihong Yu, Alexander Korotkov
2020-12-20 05:20:33 +01:00
<row>
<entry><type>anymultirange</type></entry>
<entry>Simple</entry>
<entry>Indicates that a function accepts any multirange data type
(see <xref linkend="rangetypes"/>)
</entry>
</row>
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
<row>
<entry><type>anycompatible</type></entry>
<entry>Common</entry>
<entry>Indicates that a function accepts any data type,
with automatic promotion of multiple arguments to a common data type
</entry>
</row>
<row>
<entry><type>anycompatiblearray</type></entry>
<entry>Common</entry>
<entry>Indicates that a function accepts any array data type,
with automatic promotion of multiple arguments to a common data type
</entry>
</row>
<row>
<entry><type>anycompatiblenonarray</type></entry>
<entry>Common</entry>
<entry>Indicates that a function accepts any non-array data type,
with automatic promotion of multiple arguments to a common data type
</entry>
</row>
<row>
<entry><type>anycompatiblerange</type></entry>
<entry>Common</entry>
<entry>Indicates that a function accepts any range data type,
with automatic promotion of multiple arguments to a common data type
</entry>
</row>
Multirange datatypes
Multiranges are basically sorted arrays of non-overlapping ranges with
set-theoretic operations defined over them.
Since v14, each range type automatically gets a corresponding multirange
datatype. There are both manual and automatic mechanisms for naming multirange
types. Once can specify multirange type name using multirange_type_name
attribute in CREATE TYPE. Otherwise, a multirange type name is generated
automatically. If the range type name contains "range" then we change that to
"multirange". Otherwise, we add "_multirange" to the end.
Implementation of multiranges comes with a space-efficient internal
representation format, which evades extra paddings and duplicated storage of
oids. Altogether this format allows fetching a particular range by its index
in O(n).
Statistic gathering and selectivity estimation are implemented for multiranges.
For this purpose, stored multirange is approximated as union range without gaps.
This field will likely need improvements in the future.
Catversion is bumped.
Discussion: https://postgr.es/m/CALNJ-vSUpQ_Y%3DjXvTxt1VYFztaBSsWVXeF1y6gTYQ4bOiWDLgQ%40mail.gmail.com
Discussion: https://postgr.es/m/a0b8026459d1e6167933be2104a6174e7d40d0ab.camel%40j-davis.com#fe7218c83b08068bfffb0c5293eceda0
Author: Paul Jungwirth, revised by me
Reviewed-by: David Fetter, Corey Huinker, Jeff Davis, Pavel Stehule
Reviewed-by: Alvaro Herrera, Tom Lane, Isaac Morland, David G. Johnston
Reviewed-by: Zhihong Yu, Alexander Korotkov
2020-12-20 05:20:33 +01:00
<row>
<entry><type>anycompatiblemultirange</type></entry>
<entry>Common</entry>
<entry>Indicates that a function accepts any multirange data type,
with automatic promotion of multiple arguments to a common data type
</entry>
</row>
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
</tbody>
</tgroup>
</table>
2003-08-10 00:50:22 +02:00
<para>
Polymorphic arguments and results are tied to each other and are resolved
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
to specific data types when a query calling a polymorphic function is
parsed. When there is more than one polymorphic argument, the actual
data types of the input values must match up as described below. If the
function's result type is polymorphic, or it has output parameters of
polymorphic types, the types of those results are deduced from the
actual types of the polymorphic inputs as described below.
</para>
<para>
For the <quote>simple</quote> family of polymorphic types, the
matching and deduction rules work like this:
</para>
<para>
Each position (either argument or return value) declared as
2003-08-10 00:50:22 +02:00
<type>anyelement</type> is allowed to have any specific actual
data type, but in any given call they must all be the
2010-11-23 21:27:50 +01:00
<emphasis>same</emphasis> actual type. Each
2003-08-10 00:50:22 +02:00
position declared as <type>anyarray</type> can have any array data type,
2011-11-19 00:23:55 +01:00
but similarly they must all be the same type. And similarly,
positions declared as <type>anyrange</type> must all be the same range
Multirange datatypes
Multiranges are basically sorted arrays of non-overlapping ranges with
set-theoretic operations defined over them.
Since v14, each range type automatically gets a corresponding multirange
datatype. There are both manual and automatic mechanisms for naming multirange
types. Once can specify multirange type name using multirange_type_name
attribute in CREATE TYPE. Otherwise, a multirange type name is generated
automatically. If the range type name contains "range" then we change that to
"multirange". Otherwise, we add "_multirange" to the end.
Implementation of multiranges comes with a space-efficient internal
representation format, which evades extra paddings and duplicated storage of
oids. Altogether this format allows fetching a particular range by its index
in O(n).
Statistic gathering and selectivity estimation are implemented for multiranges.
For this purpose, stored multirange is approximated as union range without gaps.
This field will likely need improvements in the future.
Catversion is bumped.
Discussion: https://postgr.es/m/CALNJ-vSUpQ_Y%3DjXvTxt1VYFztaBSsWVXeF1y6gTYQ4bOiWDLgQ%40mail.gmail.com
Discussion: https://postgr.es/m/a0b8026459d1e6167933be2104a6174e7d40d0ab.camel%40j-davis.com#fe7218c83b08068bfffb0c5293eceda0
Author: Paul Jungwirth, revised by me
Reviewed-by: David Fetter, Corey Huinker, Jeff Davis, Pavel Stehule
Reviewed-by: Alvaro Herrera, Tom Lane, Isaac Morland, David G. Johnston
Reviewed-by: Zhihong Yu, Alexander Korotkov
2020-12-20 05:20:33 +01:00
type. Likewise for <type>anymultirange</type>.
</para>
<para>
Furthermore, if there are
2003-08-10 00:50:22 +02:00
positions declared <type>anyarray</type> and others declared
<type>anyelement</type>, the actual array type in the
<type>anyarray</type> positions must be an array whose elements are
the same type appearing in the <type>anyelement</type> positions.
2017-10-09 03:44:17 +02:00
<type>anynonarray</type> is treated exactly the same as <type>anyelement</type>,
2007-06-07 01:00:50 +02:00
but adds the additional constraint that the actual type must not be
an array type.
2017-10-09 03:44:17 +02:00
<type>anyenum</type> is treated exactly the same as <type>anyelement</type>,
2007-04-02 05:49:42 +02:00
but adds the additional constraint that the actual type must
be an enum type.
2003-08-10 00:50:22 +02:00
</para>
Multirange datatypes
Multiranges are basically sorted arrays of non-overlapping ranges with
set-theoretic operations defined over them.
Since v14, each range type automatically gets a corresponding multirange
datatype. There are both manual and automatic mechanisms for naming multirange
types. Once can specify multirange type name using multirange_type_name
attribute in CREATE TYPE. Otherwise, a multirange type name is generated
automatically. If the range type name contains "range" then we change that to
"multirange". Otherwise, we add "_multirange" to the end.
Implementation of multiranges comes with a space-efficient internal
representation format, which evades extra paddings and duplicated storage of
oids. Altogether this format allows fetching a particular range by its index
in O(n).
Statistic gathering and selectivity estimation are implemented for multiranges.
For this purpose, stored multirange is approximated as union range without gaps.
This field will likely need improvements in the future.
Catversion is bumped.
Discussion: https://postgr.es/m/CALNJ-vSUpQ_Y%3DjXvTxt1VYFztaBSsWVXeF1y6gTYQ4bOiWDLgQ%40mail.gmail.com
Discussion: https://postgr.es/m/a0b8026459d1e6167933be2104a6174e7d40d0ab.camel%40j-davis.com#fe7218c83b08068bfffb0c5293eceda0
Author: Paul Jungwirth, revised by me
Reviewed-by: David Fetter, Corey Huinker, Jeff Davis, Pavel Stehule
Reviewed-by: Alvaro Herrera, Tom Lane, Isaac Morland, David G. Johnston
Reviewed-by: Zhihong Yu, Alexander Korotkov
2020-12-20 05:20:33 +01:00
<para>
Similarly, if there are positions declared <type>anyrange</type>
and others declared <type>anyelement</type> or <type>anyarray</type>,
the actual range type in the <type>anyrange</type> positions must be a
range whose subtype is the same type appearing in
the <type>anyelement</type> positions and the same as the element type
of the <type>anyarray</type> positions.
If there are positions declared <type>anymultirange</type>,
their actual multirange type must contain ranges matching parameters declared
<type>anyrange</type> and base elements matching parameters declared
<type>anyelement</type> and <type>anyarray</type>.
</para>
2003-08-10 00:50:22 +02:00
<para>
Thus, when more than one argument position is declared with a polymorphic
type, the net effect is that only certain combinations of actual argument
types are allowed. For example, a function declared as
2017-10-09 03:44:17 +02:00
<literal>equal(anyelement, anyelement)</literal> will take any two input values,
2003-08-10 00:50:22 +02:00
so long as they are of the same data type.
</para>
<para>
When the return value of a function is declared as a polymorphic type,
there must be at least one argument position that is also polymorphic,
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
and the actual data type(s) supplied for the polymorphic arguments
determine the actual
2003-08-10 00:50:22 +02:00
result type for that call. For example, if there were not already
an array subscripting mechanism, one could define a function that
implements subscripting as <literal>subscript(anyarray, integer)
2017-10-09 03:44:17 +02:00
returns anyelement</literal>. This declaration constrains the actual first
2003-08-10 00:50:22 +02:00
argument to be an array type, and allows the parser to infer the correct
2007-04-02 05:49:42 +02:00
result type from the actual first argument's type. Another example
2017-10-09 03:44:17 +02:00
is that a function declared as <literal>f(anyarray) returns anyenum</literal>
2007-04-02 05:49:42 +02:00
will only accept arrays of enum types.
2003-08-10 00:50:22 +02:00
</para>
2007-06-07 01:00:50 +02:00
2020-03-17 20:05:16 +01:00
<para>
In most cases, the parser can infer the actual data type for a
polymorphic result type from arguments that are of a different
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
polymorphic type in the same family; for example <type>anyarray</type>
can be deduced from <type>anyelement</type> or vice versa.
An exception is that a
2020-03-17 20:05:16 +01:00
polymorphic result of type <type>anyrange</type> requires an argument
of type <type>anyrange</type>; it cannot be deduced
from <type>anyarray</type> or <type>anyelement</type> arguments. This
is because there could be multiple range types with the same subtype.
</para>
2007-06-07 01:00:50 +02:00
<para>
2017-10-09 03:44:17 +02:00
Note that <type>anynonarray</type> and <type>anyenum</type> do not represent
2007-06-07 01:00:50 +02:00
separate type variables; they are the same type as
<type>anyelement</type>, just with an additional constraint. For
2017-10-09 03:44:17 +02:00
example, declaring a function as <literal>f(anyelement, anyenum)</literal>
is equivalent to declaring it as <literal>f(anyenum, anyenum)</literal>:
2007-06-07 01:00:50 +02:00
both actual arguments have to be the same enum type.
</para>
2008-07-16 03:30:23 +02:00
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
<para>
For the <quote>common</quote> family of polymorphic types, the
matching and deduction rules work approximately the same as for
the <quote>simple</quote> family, with one major difference: the
actual types of the arguments need not be identical, so long as they
can be implicitly cast to a single common type. The common type is
selected following the same rules as for <literal>UNION</literal> and
related constructs (see <xref linkend="typeconv-union-case"/>).
Selection of the common type considers the actual types
of <type>anycompatible</type> and <type>anycompatiblenonarray</type>
inputs, the array element types of <type>anycompatiblearray</type>
Multirange datatypes
Multiranges are basically sorted arrays of non-overlapping ranges with
set-theoretic operations defined over them.
Since v14, each range type automatically gets a corresponding multirange
datatype. There are both manual and automatic mechanisms for naming multirange
types. Once can specify multirange type name using multirange_type_name
attribute in CREATE TYPE. Otherwise, a multirange type name is generated
automatically. If the range type name contains "range" then we change that to
"multirange". Otherwise, we add "_multirange" to the end.
Implementation of multiranges comes with a space-efficient internal
representation format, which evades extra paddings and duplicated storage of
oids. Altogether this format allows fetching a particular range by its index
in O(n).
Statistic gathering and selectivity estimation are implemented for multiranges.
For this purpose, stored multirange is approximated as union range without gaps.
This field will likely need improvements in the future.
Catversion is bumped.
Discussion: https://postgr.es/m/CALNJ-vSUpQ_Y%3DjXvTxt1VYFztaBSsWVXeF1y6gTYQ4bOiWDLgQ%40mail.gmail.com
Discussion: https://postgr.es/m/a0b8026459d1e6167933be2104a6174e7d40d0ab.camel%40j-davis.com#fe7218c83b08068bfffb0c5293eceda0
Author: Paul Jungwirth, revised by me
Reviewed-by: David Fetter, Corey Huinker, Jeff Davis, Pavel Stehule
Reviewed-by: Alvaro Herrera, Tom Lane, Isaac Morland, David G. Johnston
Reviewed-by: Zhihong Yu, Alexander Korotkov
2020-12-20 05:20:33 +01:00
inputs, the range subtypes of <type>anycompatiblerange</type> inputs,
2021-02-24 08:13:17 +01:00
and the multirange subtypes of <type>anycompatiblemultirange</type>
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
inputs. If <type>anycompatiblenonarray</type> is present then the
common type is required to be a non-array type. Once a common type is
identified, arguments in <type>anycompatible</type>
and <type>anycompatiblenonarray</type> positions are automatically
cast to that type, and arguments in <type>anycompatiblearray</type>
positions are automatically cast to the array type for that type.
</para>
<para>
Since there is no way to select a range type knowing only its subtype,
Multirange datatypes
Multiranges are basically sorted arrays of non-overlapping ranges with
set-theoretic operations defined over them.
Since v14, each range type automatically gets a corresponding multirange
datatype. There are both manual and automatic mechanisms for naming multirange
types. Once can specify multirange type name using multirange_type_name
attribute in CREATE TYPE. Otherwise, a multirange type name is generated
automatically. If the range type name contains "range" then we change that to
"multirange". Otherwise, we add "_multirange" to the end.
Implementation of multiranges comes with a space-efficient internal
representation format, which evades extra paddings and duplicated storage of
oids. Altogether this format allows fetching a particular range by its index
in O(n).
Statistic gathering and selectivity estimation are implemented for multiranges.
For this purpose, stored multirange is approximated as union range without gaps.
This field will likely need improvements in the future.
Catversion is bumped.
Discussion: https://postgr.es/m/CALNJ-vSUpQ_Y%3DjXvTxt1VYFztaBSsWVXeF1y6gTYQ4bOiWDLgQ%40mail.gmail.com
Discussion: https://postgr.es/m/a0b8026459d1e6167933be2104a6174e7d40d0ab.camel%40j-davis.com#fe7218c83b08068bfffb0c5293eceda0
Author: Paul Jungwirth, revised by me
Reviewed-by: David Fetter, Corey Huinker, Jeff Davis, Pavel Stehule
Reviewed-by: Alvaro Herrera, Tom Lane, Isaac Morland, David G. Johnston
Reviewed-by: Zhihong Yu, Alexander Korotkov
2020-12-20 05:20:33 +01:00
use of <type>anycompatiblerange</type> and/or
<type>anycompatiblemultirange</type> requires that all arguments declared
with that type have the same actual range and/or multirange type, and that
that type's subtype agree with the selected common type, so that no casting
of the range values is required. As with <type>anyrange</type> and
<type>anymultirange</type>, use of <type>anycompatiblerange</type> and
<type>anymultirange</type> as a function result type requires that there be
an <type>anycompatiblerange</type> or <type>anycompatiblemultirange</type>
argument.
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
</para>
<para>
Notice that there is no <type>anycompatibleenum</type> type. Such a
type would not be very useful, since there normally are not any
implicit casts to enum types, meaning that there would be no way to
resolve a common type for dissimilar enum inputs.
</para>
<para>
The <quote>simple</quote> and <quote>common</quote> polymorphic
families represent two independent sets of type variables. Consider
for example
<programlisting>
CREATE FUNCTION myfunc(a anyelement, b anyelement,
c anycompatible, d anycompatible)
RETURNS anycompatible AS ...
</programlisting>
In an actual call of this function, the first two inputs must have
exactly the same type. The last two inputs must be promotable to a
common type, but this type need not have anything to do with the type
of the first two inputs. The result will have the common type of the
last two inputs.
</para>
2008-07-16 03:30:23 +02:00
<para>
A variadic function (one taking a variable number of arguments, as in
2017-11-23 15:39:47 +01:00
<xref linkend="xfunc-sql-variadic-functions"/>) can be
2008-07-16 03:30:23 +02:00
polymorphic: this is accomplished by declaring its last parameter as
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
<literal>VARIADIC</literal> <type>anyarray</type> or
<literal>VARIADIC</literal> <type>anycompatiblearray</type>.
For purposes of argument
2008-07-16 03:30:23 +02:00
matching and determining the actual result type, such a function behaves
the same as if you had written the appropriate number of
Introduce "anycompatible" family of polymorphic types.
This patch adds the pseudo-types anycompatible, anycompatiblearray,
anycompatiblenonarray, and anycompatiblerange. They work much like
anyelement, anyarray, anynonarray, and anyrange respectively, except
that the actual input values need not match precisely in type.
Instead, if we can find a common supertype (using the same rules
as for UNION/CASE type resolution), then the parser automatically
promotes the input values to that type. For example,
"myfunc(anycompatible, anycompatible)" can match a call with one
integer and one bigint argument, with the integer automatically
promoted to bigint. With anyelement in the definition, the user
would have had to cast the integer explicitly.
The new types also provide a second, independent set of type variables
for function matching; thus with "myfunc(anyelement, anyelement,
anycompatible) returns anycompatible" the first two arguments are
constrained to be the same type, but the third can be some other
type, and the result has the type of the third argument. The need
for more than one set of type variables was foreseen back when we
first invented the polymorphic types, but we never did anything
about it.
Pavel Stehule, revised a bit by me
Discussion: https://postgr.es/m/CAFj8pRDna7VqNi8gR+Tt2Ktmz0cq5G93guc3Sbn_NVPLdXAkqA@mail.gmail.com
2020-03-19 16:43:11 +01:00
<type>anynonarray</type> or <type>anycompatiblenonarray</type>
parameters.
2008-07-16 03:30:23 +02:00
</para>
2003-08-10 00:50:22 +02:00
</sect2>
1999-07-22 17:11:05 +02:00
</sect1>
2003-04-10 03:22:45 +02:00
&xfunc;
2003-08-10 00:50:22 +02:00
&xaggr;
2003-04-10 03:22:45 +02:00
&xtypes;
&xoper;
2003-04-13 11:57:35 +02:00
&xindex;
1999-07-22 17:11:05 +02:00
2010-06-01 04:31:36 +02:00
2011-02-08 22:08:41 +01:00
<sect1 id="extend-extensions">
<title>Packaging Related Objects into an Extension</title>
<indexterm zone="extend-extensions">
<primary>extension</primary>
2010-06-01 04:31:36 +02:00
</indexterm>
<para>
2017-10-09 03:44:17 +02:00
A useful extension to <productname>PostgreSQL</productname> typically includes
2011-05-19 00:14:45 +02:00
multiple SQL objects; for example, a new data type will require new
2011-02-08 22:08:41 +01:00
functions, new operators, and probably new index operator classes.
It is helpful to collect all these objects into a single package
2017-10-09 03:44:17 +02:00
to simplify database management. <productname>PostgreSQL</productname> calls
such a package an <firstterm>extension</firstterm>. To define an extension,
you need at least a <firstterm>script file</firstterm> that contains the
<acronym>SQL</acronym> commands to create the extension's objects, and a
<firstterm>control file</firstterm> that specifies a few basic properties
2011-02-08 22:08:41 +01:00
of the extension itself. If the extension includes C code, there
will typically also be a shared library file into which the C code
has been built. Once you have these files, a simple
Improve <xref> vs. <command> formatting in the documentation
SQL commands are generally marked up as <command>, except when a link
to a reference page is used using <xref>. But the latter doesn't
create monospace markup, so this looks strange especially when a
paragraph contains a mix of links and non-links.
We considered putting <command> in the <refentrytitle> on the target
side, but that creates some formatting side effects elsewhere.
Generally, it seems safer to solve this on the link source side.
We can't put the <xref> inside the <command>; the DTD doesn't allow
this. DocBook 5 would allow the <command> to have the linkend
attribute itself, but we are not there yet.
So to solve this for now, convert the <xref>s to <link> plus
<command>. This gives the correct look and also gives some more
flexibility what we can put into the link text (e.g., subcommands or
other clauses). In the future, these could then be converted to
DocBook 5 style.
I haven't converted absolutely all xrefs to SQL command reference
pages, only those where we care about the appearance of the link text
or where it was otherwise appropriate to make the appearance match a
bit better. Also in some cases, the links where repetitive, so in
those cases the links where just removed and replaced by a plain
<command>. In cases where we just want the link and don't
specifically care about the generated link text (typically phrased
"for further information see <xref ...>") the xref is kept.
Reported-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/87o8pco34z.fsf@wibble.ilmari.org
2020-10-03 16:16:51 +02:00
<link linkend="sql-createextension"><command>CREATE EXTENSION</command></link> command loads the objects into
2011-02-08 22:08:41 +01:00
your database.
</para>
<para>
2011-02-12 03:25:20 +01:00
The main advantage of using an extension, rather than just running the
2017-10-09 03:44:17 +02:00
<acronym>SQL</acronym> script to load a bunch of <quote>loose</quote> objects
into your database, is that <productname>PostgreSQL</productname> will then
2011-02-08 22:08:41 +01:00
understand that the objects of the extension go together. You can
Improve <xref> vs. <command> formatting in the documentation
SQL commands are generally marked up as <command>, except when a link
to a reference page is used using <xref>. But the latter doesn't
create monospace markup, so this looks strange especially when a
paragraph contains a mix of links and non-links.
We considered putting <command> in the <refentrytitle> on the target
side, but that creates some formatting side effects elsewhere.
Generally, it seems safer to solve this on the link source side.
We can't put the <xref> inside the <command>; the DTD doesn't allow
this. DocBook 5 would allow the <command> to have the linkend
attribute itself, but we are not there yet.
So to solve this for now, convert the <xref>s to <link> plus
<command>. This gives the correct look and also gives some more
flexibility what we can put into the link text (e.g., subcommands or
other clauses). In the future, these could then be converted to
DocBook 5 style.
I haven't converted absolutely all xrefs to SQL command reference
pages, only those where we care about the appearance of the link text
or where it was otherwise appropriate to make the appearance match a
bit better. Also in some cases, the links where repetitive, so in
those cases the links where just removed and replaced by a plain
<command>. In cases where we just want the link and don't
specifically care about the generated link text (typically phrased
"for further information see <xref ...>") the xref is kept.
Reported-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/87o8pco34z.fsf@wibble.ilmari.org
2020-10-03 16:16:51 +02:00
drop all the objects with a single <link linkend="sql-dropextension"><command>DROP EXTENSION</command></link>
2017-10-09 03:44:17 +02:00
command (no need to maintain a separate <quote>uninstall</quote> script).
Even more useful, <application>pg_dump</application> knows that it should not
2011-02-08 22:08:41 +01:00
dump the individual member objects of the extension — it will
2017-10-09 03:44:17 +02:00
just include a <command>CREATE EXTENSION</command> command in dumps, instead.
2011-02-08 22:08:41 +01:00
This vastly simplifies migration to a new version of the extension
that might contain more or different objects than the old version.
Note however that you must have the extension's control, script, and
other files available when loading such a dump into a new database.
</para>
<para>
2017-10-09 03:44:17 +02:00
<productname>PostgreSQL</productname> will not let you drop an individual object
2011-02-08 22:08:41 +01:00
contained in an extension, except by dropping the whole extension.
Also, while you can change the definition of an extension member object
(for example, via <command>CREATE OR REPLACE FUNCTION</command> for a
function), bear in mind that the modified definition will not be dumped
2017-10-09 03:44:17 +02:00
by <application>pg_dump</application>. Such a change is usually only sensible if
2011-02-08 22:08:41 +01:00
you concurrently make the same change in the extension's script file.
(But there are special provisions for tables containing configuration
2017-11-23 15:39:47 +01:00
data; see <xref linkend="extend-extensions-config-tables"/>.)
2016-09-07 19:36:08 +02:00
In production situations, it's generally better to create an extension
update script to perform changes to extension member objects.
2011-02-08 22:08:41 +01:00
</para>
2016-04-07 03:45:32 +02:00
<para>
2016-09-07 19:36:08 +02:00
The extension script may set privileges on objects that are part of the
Make contrib modules' installation scripts more secure.
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
2020-08-10 16:44:42 +02:00
extension, using <command>GRANT</command> and <command>REVOKE</command>
2016-04-07 03:45:32 +02:00
statements. The final set of privileges for each object (if any are set)
will be stored in the
<link linkend="catalog-pg-init-privs"><structname>pg_init_privs</structname></link>
2017-10-09 03:44:17 +02:00
system catalog. When <application>pg_dump</application> is used, the
<command>CREATE EXTENSION</command> command will be included in the dump, followed
2016-04-07 03:45:32 +02:00
by the set of <command>GRANT</command> and <command>REVOKE</command>
statements necessary to set the privileges on the objects to what they were
at the time the dump was taken.
</para>
<para>
2017-10-09 03:44:17 +02:00
<productname>PostgreSQL</productname> does not currently support extension scripts
2016-04-07 03:45:32 +02:00
issuing <command>CREATE POLICY</command> or <command>SECURITY LABEL</command>
statements. These are expected to be set after the extension has been
created. All RLS policies and security labels on extension objects will be
2017-10-09 03:44:17 +02:00
included in dumps created by <application>pg_dump</application>.
2016-04-07 03:45:32 +02:00
</para>
2011-02-12 03:25:20 +01:00
<para>
The extension mechanism also has provisions for packaging modification
scripts that adjust the definitions of the SQL objects contained in an
extension. For example, if version 1.1 of an extension adds one function
and changes the body of another function compared to 1.0, the extension
2017-10-09 03:44:17 +02:00
author can provide an <firstterm>update script</firstterm> that makes just those
two changes. The <command>ALTER EXTENSION UPDATE</command> command can then
2011-02-12 03:25:20 +01:00
be used to apply these changes and track which version of the extension
is actually installed in a given database.
</para>
2011-02-09 17:55:32 +01:00
<para>
The kinds of SQL objects that can be members of an extension are shown in
Improve <xref> vs. <command> formatting in the documentation
SQL commands are generally marked up as <command>, except when a link
to a reference page is used using <xref>. But the latter doesn't
create monospace markup, so this looks strange especially when a
paragraph contains a mix of links and non-links.
We considered putting <command> in the <refentrytitle> on the target
side, but that creates some formatting side effects elsewhere.
Generally, it seems safer to solve this on the link source side.
We can't put the <xref> inside the <command>; the DTD doesn't allow
this. DocBook 5 would allow the <command> to have the linkend
attribute itself, but we are not there yet.
So to solve this for now, convert the <xref>s to <link> plus
<command>. This gives the correct look and also gives some more
flexibility what we can put into the link text (e.g., subcommands or
other clauses). In the future, these could then be converted to
DocBook 5 style.
I haven't converted absolutely all xrefs to SQL command reference
pages, only those where we care about the appearance of the link text
or where it was otherwise appropriate to make the appearance match a
bit better. Also in some cases, the links where repetitive, so in
those cases the links where just removed and replaced by a plain
<command>. In cases where we just want the link and don't
specifically care about the generated link text (typically phrased
"for further information see <xref ...>") the xref is kept.
Reported-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/87o8pco34z.fsf@wibble.ilmari.org
2020-10-03 16:16:51 +02:00
the description of <link linkend="sql-alterextension"><command>ALTER EXTENSION</command></link>. Notably, objects
2011-02-09 17:55:32 +01:00
that are database-cluster-wide, such as databases, roles, and tablespaces,
cannot be extension members since an extension is only known within one
database. (Although an extension script is not prohibited from creating
such objects, if it does so they will not be tracked as part of the
extension.) Also notice that while a table can be a member of an
extension, its subsidiary objects such as indexes are not directly
considered members of the extension.
2013-04-05 04:37:25 +02:00
Another important point is that schemas can belong to extensions, but not
vice versa: an extension as such has an unqualified name and does not
2017-10-09 03:44:17 +02:00
exist <quote>within</quote> any schema. The extension's member objects,
2013-04-05 04:37:25 +02:00
however, will belong to schemas whenever appropriate for their object
types. It may or may not be appropriate for an extension to own the
schema(s) its member objects are within.
2011-02-09 17:55:32 +01:00
</para>
Delete deleteWhatDependsOn() in favor of more performDeletion() flag bits.
deleteWhatDependsOn() had grown an uncomfortably large number of
assumptions about what it's used for. There are actually only two minor
differences between what it does and what a regular performDeletion() call
can do, so let's invent additional bits in performDeletion's existing flags
argument that specify those behaviors, and get rid of deleteWhatDependsOn()
as such. (We'd probably have done it this way from the start, except that
performDeletion didn't originally have a flags argument, IIRC.)
Also, add a SKIP_EXTENSIONS flag bit that prevents ever recursing to an
extension, and use that when dropping temporary objects at session end.
This provides a more general solution to the problem addressed in a hacky
way in commit 08dd23cec: if an extension script creates temp objects and
forgets to remove them again, the whole extension went away when its
contained temp objects were deleted. The previous solution only covered
temp relations, but this solves it for all object types.
These changes require minor additions in dependency.c to pass the flags
to subroutines that previously didn't get them, but it's still a net
savings of code, and it seems cleaner than before.
Having done this, revert the special-case code added in 08dd23cec that
prevented addition of pg_depend records for temp table extension
membership, because that caused its own oddities: dropping an extension
that had created such a table didn't automatically remove the table,
leading to a failure if the table had another dependency on the extension
(such as use of an extension data type), or to a duplicate-name failure if
you then tried to recreate the extension. But we keep the part that
prevents the pg_temp_nnn schema from becoming an extension member; we never
want that to happen. Add a regression test case covering these behaviors.
Although this fixes some arguable bugs, we've heard few field complaints,
and any such problems are easily worked around by explicitly dropping temp
objects at the end of extension scripts (which seems like good practice
anyway). So I won't risk a back-patch.
Discussion: https://postgr.es/m/e51f4311-f483-4dd0-1ccc-abec3c405110@BlueTreble.com
2016-12-02 20:57:35 +01:00
<para>
If an extension's script creates any temporary objects (such as temp
tables), those objects are treated as extension members for the
remainder of the current session, but are automatically dropped at
session end, as any temporary object would be. This is an exception
to the rule that extension member objects cannot be dropped without
dropping the whole extension.
</para>
2023-01-09 21:08:24 +01:00
<sect2 id="extend-extensions-files">
2011-02-08 22:08:41 +01:00
<title>Extension Files</title>
<indexterm>
<primary>control file</primary>
</indexterm>
<para>
Improve <xref> vs. <command> formatting in the documentation
SQL commands are generally marked up as <command>, except when a link
to a reference page is used using <xref>. But the latter doesn't
create monospace markup, so this looks strange especially when a
paragraph contains a mix of links and non-links.
We considered putting <command> in the <refentrytitle> on the target
side, but that creates some formatting side effects elsewhere.
Generally, it seems safer to solve this on the link source side.
We can't put the <xref> inside the <command>; the DTD doesn't allow
this. DocBook 5 would allow the <command> to have the linkend
attribute itself, but we are not there yet.
So to solve this for now, convert the <xref>s to <link> plus
<command>. This gives the correct look and also gives some more
flexibility what we can put into the link text (e.g., subcommands or
other clauses). In the future, these could then be converted to
DocBook 5 style.
I haven't converted absolutely all xrefs to SQL command reference
pages, only those where we care about the appearance of the link text
or where it was otherwise appropriate to make the appearance match a
bit better. Also in some cases, the links where repetitive, so in
those cases the links where just removed and replaced by a plain
<command>. In cases where we just want the link and don't
specifically care about the generated link text (typically phrased
"for further information see <xref ...>") the xref is kept.
Reported-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/87o8pco34z.fsf@wibble.ilmari.org
2020-10-03 16:16:51 +02:00
The <command>CREATE EXTENSION</command> command relies on a control
2011-02-08 22:08:41 +01:00
file for each extension, which must be named the same as the extension
2017-10-09 03:44:17 +02:00
with a suffix of <literal>.control</literal>, and must be placed in the
2011-02-12 04:53:43 +01:00
installation's <literal>SHAREDIR/extension</literal> directory. There
2017-10-09 03:44:17 +02:00
must also be at least one <acronym>SQL</acronym> script file, which follows the
2011-02-12 03:25:20 +01:00
naming pattern
2017-10-09 03:44:17 +02:00
<literal><replaceable>extension</replaceable>--<replaceable>version</replaceable>.sql</literal>
(for example, <literal>foo--1.0.sql</literal> for version <literal>1.0</literal> of
extension <literal>foo</literal>). By default, the script file(s) are also
2011-02-12 04:53:43 +01:00
placed in the <literal>SHAREDIR/extension</literal> directory; but the
2011-02-12 03:25:20 +01:00
control file can specify a different directory for the script file(s).
2011-02-08 22:08:41 +01:00
</para>
<para>
The file format for an extension control file is the same as for the
2017-10-09 03:44:17 +02:00
<filename>postgresql.conf</filename> file, namely a list of
<replaceable>parameter_name</replaceable> <literal>=</literal> <replaceable>value</replaceable>
2011-02-08 22:08:41 +01:00
assignments, one per line. Blank lines and comments introduced by
2017-10-09 03:44:17 +02:00
<literal>#</literal> are allowed. Be sure to quote any value that is not
2011-02-08 22:08:41 +01:00
a single word or number.
</para>
<para>
A control file can set the following parameters:
</para>
<variablelist>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-extensions-files-directory">
2011-02-12 03:25:20 +01:00
<term><varname>directory</varname> (<type>string</type>)</term>
2011-02-08 22:08:41 +01:00
<listitem>
<para>
2017-10-09 03:44:17 +02:00
The directory containing the extension's <acronym>SQL</acronym> script
2011-02-12 03:25:20 +01:00
file(s). Unless an absolute path is given, the name is relative to
2011-02-12 04:53:43 +01:00
the installation's <literal>SHAREDIR</literal> directory. The
default behavior is equivalent to specifying
2017-10-09 03:44:17 +02:00
<literal>directory = 'extension'</literal>.
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-extensions-files-default-version">
2011-02-12 03:25:20 +01:00
<term><varname>default_version</varname> (<type>string</type>)</term>
2011-02-08 22:08:41 +01:00
<listitem>
<para>
2011-02-12 03:25:20 +01:00
The default version of the extension (the one that will be installed
2017-10-09 03:44:17 +02:00
if no version is specified in <command>CREATE EXTENSION</command>). Although
this can be omitted, that will result in <command>CREATE EXTENSION</command>
failing if no <literal>VERSION</literal> option appears, so you generally
2011-02-12 03:25:20 +01:00
don't want to do that.
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-extensions-files-comment">
2011-02-08 22:08:41 +01:00
<term><varname>comment</varname> (<type>string</type>)</term>
<listitem>
<para>
2016-09-07 19:36:08 +02:00
A comment (any string) about the extension. The comment is applied
when initially creating an extension, but not during extension updates
(since that might override user-added comments). Alternatively,
the extension's comment can be set by writing
2017-11-23 15:39:47 +01:00
a <xref linkend="sql-comment"/> command in the script file.
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-extensions-files-encoding">
2011-02-13 19:38:05 +01:00
<term><varname>encoding</varname> (<type>string</type>)</term>
2011-02-08 22:08:41 +01:00
<listitem>
<para>
2011-02-13 19:38:05 +01:00
The character set encoding used by the script file(s). This should
be specified if the script files contain any non-ASCII characters.
Otherwise the files will be assumed to be in the database encoding.
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-extensions-files-module-pathname">
2011-02-13 19:38:05 +01:00
<term><varname>module_pathname</varname> (<type>string</type>)</term>
2011-02-08 22:08:41 +01:00
<listitem>
<para>
2011-02-13 19:38:05 +01:00
The value of this parameter will be substituted for each occurrence
2017-10-09 03:44:17 +02:00
of <literal>MODULE_PATHNAME</literal> in the script file(s). If it is not
2011-02-13 19:38:05 +01:00
set, no substitution is made. Typically, this is set to
2017-10-09 03:44:17 +02:00
<literal>$libdir/<replaceable>shared_library_name</replaceable></literal> and
then <literal>MODULE_PATHNAME</literal> is used in <command>CREATE
FUNCTION</command> commands for C-language functions, so that the script
2011-02-13 19:38:05 +01:00
files do not need to hard-wire the name of the shared library.
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-extensions-files-requires">
2011-02-13 19:38:05 +01:00
<term><varname>requires</varname> (<type>string</type>)</term>
<listitem>
<para>
A list of names of extensions that this extension depends on,
for example <literal>requires = 'foo, bar'</literal>. Those
extensions must be installed before this one can be installed.
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
2023-03-20 23:37:11 +01:00
<varlistentry id="extend-extensions-files-no-relocate">
<term><varname>no_relocate</varname> (<type>string</type>)</term>
<listitem>
<para>
A list of names of extensions that this extension depends on that
should be barred from changing their schemas via <command>ALTER
EXTENSION ... SET SCHEMA</command>.
This is needed if this extension's script references the name
of a required extension's schema (using
the <literal>@extschema:<replaceable>name</replaceable>@</literal>
syntax) in a way that cannot track renames.
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-extensions-files-superuser">
2011-03-04 22:08:24 +01:00
<term><varname>superuser</varname> (<type>boolean</type>)</term>
<listitem>
<para>
2017-10-09 03:44:17 +02:00
If this parameter is <literal>true</literal> (which is the default),
2011-03-04 22:08:24 +01:00
only superusers can create the extension or update it to a new
Make contrib modules' installation scripts more secure.
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
2020-08-10 16:44:42 +02:00
version (but see also <varname>trusted</varname>, below).
If it is set to <literal>false</literal>, just the privileges
2011-03-04 22:08:24 +01:00
required to execute the commands in the installation or update script
are required.
Invent "trusted" extensions, and remove the pg_pltemplate catalog.
This patch creates a new extension property, "trusted". An extension
that's marked that way in its control file can be installed by a
non-superuser who has the CREATE privilege on the current database,
even if the extension contains objects that normally would have to be
created by a superuser. The objects within the extension will (by
default) be owned by the bootstrap superuser, but the extension itself
will be owned by the calling user. This allows replicating the old
behavior around trusted procedural languages, without all the
special-case logic in CREATE LANGUAGE. We have, however, chosen to
loosen the rules slightly: formerly, only a database owner could take
advantage of the special case that allowed installation of a trusted
language, but now anyone who has CREATE privilege can do so.
Having done that, we can delete the pg_pltemplate catalog, moving the
knowledge it contained into the extension script files for the various
PLs. This ends up being no change at all for the in-core PLs, but it is
a large step forward for external PLs: they can now have the same ease
of installation as core PLs do. The old "trusted PL" behavior was only
available to PLs that had entries in pg_pltemplate, but now any
extension can be marked trusted if appropriate.
This also removes one of the stumbling blocks for our Python 2 -> 3
migration, since the association of "plpythonu" with Python 2 is no
longer hard-wired into pg_pltemplate's initial contents. Exactly where
we go from here on that front remains to be settled, but one problem
is fixed.
Patch by me, reviewed by Peter Eisentraut, Stephen Frost, and others.
Discussion: https://postgr.es/m/5889.1566415762@sss.pgh.pa.us
2020-01-30 00:42:43 +01:00
This should normally be set to <literal>true</literal> if any of the
script commands require superuser privileges. (Such commands would
fail anyway, but it's more user-friendly to give the error up front.)
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-extensions-files-trusted">
Invent "trusted" extensions, and remove the pg_pltemplate catalog.
This patch creates a new extension property, "trusted". An extension
that's marked that way in its control file can be installed by a
non-superuser who has the CREATE privilege on the current database,
even if the extension contains objects that normally would have to be
created by a superuser. The objects within the extension will (by
default) be owned by the bootstrap superuser, but the extension itself
will be owned by the calling user. This allows replicating the old
behavior around trusted procedural languages, without all the
special-case logic in CREATE LANGUAGE. We have, however, chosen to
loosen the rules slightly: formerly, only a database owner could take
advantage of the special case that allowed installation of a trusted
language, but now anyone who has CREATE privilege can do so.
Having done that, we can delete the pg_pltemplate catalog, moving the
knowledge it contained into the extension script files for the various
PLs. This ends up being no change at all for the in-core PLs, but it is
a large step forward for external PLs: they can now have the same ease
of installation as core PLs do. The old "trusted PL" behavior was only
available to PLs that had entries in pg_pltemplate, but now any
extension can be marked trusted if appropriate.
This also removes one of the stumbling blocks for our Python 2 -> 3
migration, since the association of "plpythonu" with Python 2 is no
longer hard-wired into pg_pltemplate's initial contents. Exactly where
we go from here on that front remains to be settled, but one problem
is fixed.
Patch by me, reviewed by Peter Eisentraut, Stephen Frost, and others.
Discussion: https://postgr.es/m/5889.1566415762@sss.pgh.pa.us
2020-01-30 00:42:43 +01:00
<term><varname>trusted</varname> (<type>boolean</type>)</term>
<listitem>
<para>
This parameter, if set to <literal>true</literal> (which is not the
default), allows some non-superusers to install an extension that
has <varname>superuser</varname> set to <literal>true</literal>.
Specifically, installation will be permitted for anyone who has
<literal>CREATE</literal> privilege on the current database.
When the user executing <command>CREATE EXTENSION</command> is not
a superuser but is allowed to install by virtue of this parameter,
then the installation or update script is run as the bootstrap
superuser, not as the calling user.
This parameter is irrelevant if <varname>superuser</varname> is
<literal>false</literal>.
Generally, this should not be set true for extensions that could
allow access to otherwise-superuser-only abilities, such as
2020-07-05 15:37:57 +02:00
file system access.
Make contrib modules' installation scripts more secure.
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
2020-08-10 16:44:42 +02:00
Also, marking an extension trusted requires significant extra effort
to write the extension's installation and update script(s) securely;
see <xref linkend="extend-extensions-security"/>.
2011-03-04 22:08:24 +01:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-extensions-files-relocatable">
2011-02-08 22:08:41 +01:00
<term><varname>relocatable</varname> (<type>boolean</type>)</term>
<listitem>
<para>
2017-10-09 03:44:17 +02:00
An extension is <firstterm>relocatable</firstterm> if it is possible to move
2011-02-08 22:08:41 +01:00
its contained objects into a different schema after initial creation
2020-09-01 00:33:37 +02:00
of the extension. The default is <literal>false</literal>, i.e., the
2011-02-08 22:08:41 +01:00
extension is not relocatable.
2017-11-23 15:39:47 +01:00
See <xref linkend="extend-extensions-relocation"/> for more information.
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-extensions-files-schema">
2011-02-08 22:08:41 +01:00
<term><varname>schema</varname> (<type>string</type>)</term>
<listitem>
<para>
This parameter can only be set for non-relocatable extensions.
It forces the extension to be loaded into exactly the named schema
2016-09-07 19:36:08 +02:00
and not any other.
The <varname>schema</varname> parameter is consulted only when
initially creating an extension, not during extension updates.
2017-11-23 15:39:47 +01:00
See <xref linkend="extend-extensions-relocation"/> for more information.
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
2011-02-12 03:25:20 +01:00
In addition to the primary control file
2017-10-09 03:44:17 +02:00
<literal><replaceable>extension</replaceable>.control</literal>,
2011-02-12 03:25:20 +01:00
an extension can have secondary control files named in the style
2017-10-09 03:44:17 +02:00
<literal><replaceable>extension</replaceable>--<replaceable>version</replaceable>.control</literal>.
2011-02-12 03:25:20 +01:00
If supplied, these must be located in the script file directory.
Secondary control files follow the same format as the primary control
file. Any parameters set in a secondary control file override the
primary control file when installing or updating to that version of
2017-10-09 03:44:17 +02:00
the extension. However, the parameters <varname>directory</varname> and
<varname>default_version</varname> cannot be set in a secondary control file.
2011-02-12 03:25:20 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
An extension's <acronym>SQL</acronym> script files can contain any SQL commands,
except for transaction control commands (<command>BEGIN</command>,
2022-05-19 09:27:34 +02:00
<command>COMMIT</command>, etc.) and commands that cannot be executed inside a
2017-10-09 03:44:17 +02:00
transaction block (such as <command>VACUUM</command>). This is because the
2011-02-12 03:25:20 +01:00
script files are implicitly executed within a transaction block.
2011-02-08 22:08:41 +01:00
</para>
2011-10-12 21:45:03 +02:00
<para>
2017-10-09 03:44:17 +02:00
An extension's <acronym>SQL</acronym> script files can also contain lines
beginning with <literal>\echo</literal>, which will be ignored (treated as
2011-10-12 21:45:03 +02:00
comments) by the extension mechanism. This provision is commonly used
2017-10-09 03:44:17 +02:00
to throw an error if the script file is fed to <application>psql</application>
rather than being loaded via <command>CREATE EXTENSION</command> (see example
2017-11-23 15:39:47 +01:00
script in <xref linkend="extend-extensions-example"/>).
2016-09-07 19:36:08 +02:00
Without that, users might accidentally load the
2017-10-09 03:44:17 +02:00
extension's contents as <quote>loose</quote> objects rather than as an
2011-10-12 21:45:03 +02:00
extension, a state of affairs that's a bit tedious to recover from.
</para>
Invent "trusted" extensions, and remove the pg_pltemplate catalog.
This patch creates a new extension property, "trusted". An extension
that's marked that way in its control file can be installed by a
non-superuser who has the CREATE privilege on the current database,
even if the extension contains objects that normally would have to be
created by a superuser. The objects within the extension will (by
default) be owned by the bootstrap superuser, but the extension itself
will be owned by the calling user. This allows replicating the old
behavior around trusted procedural languages, without all the
special-case logic in CREATE LANGUAGE. We have, however, chosen to
loosen the rules slightly: formerly, only a database owner could take
advantage of the special case that allowed installation of a trusted
language, but now anyone who has CREATE privilege can do so.
Having done that, we can delete the pg_pltemplate catalog, moving the
knowledge it contained into the extension script files for the various
PLs. This ends up being no change at all for the in-core PLs, but it is
a large step forward for external PLs: they can now have the same ease
of installation as core PLs do. The old "trusted PL" behavior was only
available to PLs that had entries in pg_pltemplate, but now any
extension can be marked trusted if appropriate.
This also removes one of the stumbling blocks for our Python 2 -> 3
migration, since the association of "plpythonu" with Python 2 is no
longer hard-wired into pg_pltemplate's initial contents. Exactly where
we go from here on that front remains to be settled, but one problem
is fixed.
Patch by me, reviewed by Peter Eisentraut, Stephen Frost, and others.
Discussion: https://postgr.es/m/5889.1566415762@sss.pgh.pa.us
2020-01-30 00:42:43 +01:00
<para>
If the extension script contains the
string <literal>@extowner@</literal>, that string is replaced with the
(suitably quoted) name of the user calling <command>CREATE
EXTENSION</command> or <command>ALTER EXTENSION</command>. Typically
this feature is used by extensions that are marked trusted to assign
ownership of selected objects to the calling user rather than the
bootstrap superuser. (One should be careful about doing so, however.
For example, assigning ownership of a C-language function to a
non-superuser would create a privilege escalation path for that user.)
</para>
2011-02-08 22:08:41 +01:00
<para>
2011-02-12 03:25:20 +01:00
While the script files can contain any characters allowed by the specified
encoding, control files should contain only plain ASCII, because there
2017-10-09 03:44:17 +02:00
is no way for <productname>PostgreSQL</productname> to know what encoding a
2011-02-08 22:08:41 +01:00
control file is in. In practice this is only an issue if you want to
use non-ASCII characters in the extension's comment. Recommended
2017-10-09 03:44:17 +02:00
practice in that case is to not use the control file <varname>comment</varname>
parameter, but instead use <command>COMMENT ON EXTENSION</command>
2011-02-12 03:25:20 +01:00
within a script file to set the comment.
2011-02-08 22:08:41 +01:00
</para>
</sect2>
2016-09-07 19:36:08 +02:00
<sect2 id="extend-extensions-relocation">
2011-02-08 22:08:41 +01:00
<title>Extension Relocatability</title>
<para>
Users often wish to load the objects contained in an extension into a
different schema than the extension's author had in mind. There are
three supported levels of relocatability:
</para>
2010-06-01 04:31:36 +02:00
<itemizedlist>
<listitem>
<para>
2011-02-08 22:08:41 +01:00
A fully relocatable extension can be moved into another schema
at any time, even after it's been loaded into a database.
2017-10-09 03:44:17 +02:00
This is done with the <command>ALTER EXTENSION SET SCHEMA</command>
2011-02-08 22:08:41 +01:00
command, which automatically renames all the member objects into
the new schema. Normally, this is only possible if the extension
contains no internal assumptions about what schema any of its
objects are in. Also, the extension's objects must all be in one
schema to begin with (ignoring objects that do not belong to any
schema, such as procedural languages). Mark a fully relocatable
2017-10-09 03:44:17 +02:00
extension by setting <literal>relocatable = true</literal> in its control
2011-02-08 22:08:41 +01:00
file.
2010-06-03 16:41:25 +02:00
</para>
2010-06-01 04:31:36 +02:00
</listitem>
2011-02-08 22:08:41 +01:00
2010-06-01 04:31:36 +02:00
<listitem>
<para>
2011-02-08 22:08:41 +01:00
An extension might be relocatable during installation but not
afterwards. This is typically the case if the extension's script
file needs to reference the target schema explicitly, for example
2017-10-09 03:44:17 +02:00
in setting <literal>search_path</literal> properties for SQL functions.
For such an extension, set <literal>relocatable = false</literal> in its
control file, and use <literal>@extschema@</literal> to refer to the target
2011-02-08 22:08:41 +01:00
schema in the script file. All occurrences of this string will be
2023-03-20 23:37:11 +01:00
replaced by the actual target schema's name (double-quoted if
necessary) before the script is executed. The user can set the
target schema using the
2017-10-09 03:44:17 +02:00
<literal>SCHEMA</literal> option of <command>CREATE EXTENSION</command>.
2010-06-01 04:31:36 +02:00
</para>
</listitem>
2011-02-08 22:08:41 +01:00
2010-06-01 04:31:36 +02:00
<listitem>
<para>
2011-02-08 22:08:41 +01:00
If the extension does not support relocation at all, set
2017-10-09 03:44:17 +02:00
<literal>relocatable = false</literal> in its control file, and also set
<literal>schema</literal> to the name of the intended target schema. This
will prevent use of the <literal>SCHEMA</literal> option of <command>CREATE
EXTENSION</command>, unless it specifies the same schema named in the control
2011-02-08 22:08:41 +01:00
file. This choice is typically necessary if the extension contains
2023-03-20 23:37:11 +01:00
internal assumptions about its schema name that can't be replaced by
2017-10-09 03:44:17 +02:00
uses of <literal>@extschema@</literal>. The <literal>@extschema@</literal>
2011-02-08 22:08:41 +01:00
substitution mechanism is available in this case too, although it is
of limited use since the schema name is determined by the control file.
2010-06-01 04:31:36 +02:00
</para>
</listitem>
</itemizedlist>
2011-02-08 22:08:41 +01:00
<para>
In all cases, the script file will be executed with
2017-11-23 15:39:47 +01:00
<xref linkend="guc-search-path"/> initially set to point to the target
2017-10-09 03:44:17 +02:00
schema; that is, <command>CREATE EXTENSION</command> does the equivalent of
2011-02-08 22:08:41 +01:00
this:
<programlisting>
Make contrib modules' installation scripts more secure.
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
2020-08-10 16:44:42 +02:00
SET LOCAL search_path TO @extschema@, pg_temp;
2011-02-08 22:08:41 +01:00
</programlisting>
This allows the objects created by the script file to go into the target
2017-10-09 03:44:17 +02:00
schema. The script file can change <varname>search_path</varname> if it wishes,
but that is generally undesirable. <varname>search_path</varname> is restored
to its previous setting upon completion of <command>CREATE EXTENSION</command>.
2011-02-08 22:08:41 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
The target schema is determined by the <varname>schema</varname> parameter in
the control file if that is given, otherwise by the <literal>SCHEMA</literal>
option of <command>CREATE EXTENSION</command> if that is given, otherwise the
2011-02-08 22:08:41 +01:00
current default object creation schema (the first one in the caller's
2017-10-09 03:44:17 +02:00
<varname>search_path</varname>). When the control file <varname>schema</varname>
2011-02-08 22:08:41 +01:00
parameter is used, the target schema will be created if it doesn't
already exist, but in the other two cases it must already exist.
</para>
<para>
If any prerequisite extensions are listed in <varname>requires</varname>
Make contrib modules' installation scripts more secure.
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
2020-08-10 16:44:42 +02:00
in the control file, their target schemas are added to the initial
setting of <varname>search_path</varname>, following the new
extension's target schema. This allows their objects to be visible to
the new extension's script file.
</para>
<para>
For security, <literal>pg_temp</literal> is automatically appended to
the end of <varname>search_path</varname> in all cases.
2011-02-08 22:08:41 +01:00
</para>
<para>
Although a non-relocatable extension can contain objects spread across
multiple schemas, it is usually desirable to place all the objects meant
for external use into a single schema, which is considered the extension's
target schema. Such an arrangement works conveniently with the default
2017-10-09 03:44:17 +02:00
setting of <varname>search_path</varname> during creation of dependent
2011-02-08 22:08:41 +01:00
extensions.
</para>
2023-03-20 23:37:11 +01:00
<para>
If an extension references objects belonging to another extension,
it is recommended to schema-qualify those references. To do that,
write <literal>@extschema:<replaceable>name</replaceable>@</literal>
in the extension's script file, where <replaceable>name</replaceable>
is the name of the other extension (which must be listed in this
extension's <literal>requires</literal> list). This string will be
replaced by the name (double-quoted if necessary) of that extension's
target schema.
Although this notation avoids the need to make hard-wired assumptions
about schema names in the extension's script file, its use may embed
the other extension's schema name into the installed objects of this
extension. (Typically, that happens
when <literal>@extschema:<replaceable>name</replaceable>@</literal> is
used inside a string literal, such as a function body or
a <varname>search_path</varname> setting. In other cases, the object
reference is reduced to an OID during parsing and does not require
subsequent lookups.) If the other extension's schema name is so
embedded, you should prevent the other extension from being relocated
after yours is installed, by adding the name of the other extension to
this one's <literal>no_relocate</literal> list.
</para>
2011-02-08 22:08:41 +01:00
</sect2>
2016-09-07 19:36:08 +02:00
<sect2 id="extend-extensions-config-tables">
2011-02-08 22:08:41 +01:00
<title>Extension Configuration Tables</title>
<para>
Some extensions include configuration tables, which contain data that
might be added or changed by the user after installation of the
extension. Ordinarily, if a table is part of an extension, neither
the table's definition nor its content will be dumped by
2017-10-09 03:44:17 +02:00
<application>pg_dump</application>. But that behavior is undesirable for a
2011-02-08 22:08:41 +01:00
configuration table; any data changes made by the user need to be
included in dumps, or the extension will behave differently after a dump
2022-07-21 20:55:23 +02:00
and restore.
2011-02-08 22:08:41 +01:00
</para>
2012-12-20 22:30:59 +01:00
<indexterm>
<primary>pg_extension_config_dump</primary>
</indexterm>
2011-02-08 22:08:41 +01:00
<para>
To solve this problem, an extension's script file can mark a table
2016-04-18 05:13:45 +02:00
or a sequence it has created as a configuration relation, which will
2017-10-09 03:44:17 +02:00
cause <application>pg_dump</application> to include the table's or the sequence's
2016-04-18 05:13:45 +02:00
contents (not its definition) in dumps. To do that, call the function
2017-10-09 03:44:17 +02:00
<function>pg_extension_config_dump(regclass, text)</function> after creating the
2016-04-18 05:13:45 +02:00
table or the sequence, for example
2011-02-08 22:08:41 +01:00
<programlisting>
CREATE TABLE my_config (key text, value text);
2016-04-18 05:13:45 +02:00
CREATE SEQUENCE my_config_seq;
2011-02-08 22:08:41 +01:00
SELECT pg_catalog.pg_extension_config_dump('my_config', '');
2016-04-18 05:13:45 +02:00
SELECT pg_catalog.pg_extension_config_dump('my_config_seq', '');
2011-02-08 22:08:41 +01:00
</programlisting>
2016-04-18 05:13:45 +02:00
Any number of tables or sequences can be marked this way. Sequences
2017-10-09 03:44:17 +02:00
associated with <type>serial</type> or <type>bigserial</type> columns can
2016-04-18 05:13:45 +02:00
be marked as well.
2011-02-08 22:08:41 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
When the second argument of <function>pg_extension_config_dump</function> is
2011-02-08 22:08:41 +01:00
an empty string, the entire contents of the table are dumped by
2017-10-09 03:44:17 +02:00
<application>pg_dump</application>. This is usually only correct if the table
2011-02-08 22:08:41 +01:00
is initially empty as created by the extension script. If there is
a mixture of initial data and user-provided data in the table,
2017-10-09 03:44:17 +02:00
the second argument of <function>pg_extension_config_dump</function> provides
a <literal>WHERE</literal> condition that selects the data to be dumped.
2011-02-08 22:08:41 +01:00
For example, you might do
<programlisting>
CREATE TABLE my_config (key text, value text, standard_entry boolean);
SELECT pg_catalog.pg_extension_config_dump('my_config', 'WHERE NOT standard_entry');
</programlisting>
2017-10-09 03:44:17 +02:00
and then make sure that <structfield>standard_entry</structfield> is true only
2011-02-08 22:08:41 +01:00
in the rows created by the extension's script.
</para>
2016-04-18 05:13:45 +02:00
<para>
2017-10-09 03:44:17 +02:00
For sequences, the second argument of <function>pg_extension_config_dump</function>
2016-04-18 05:13:45 +02:00
has no effect.
</para>
2011-02-08 22:08:41 +01:00
<para>
More complicated situations, such as initially-provided rows that might
be modified by users, can be handled by creating triggers on the
configuration table to ensure that modified rows are marked correctly.
</para>
2012-12-20 22:30:59 +01:00
<para>
You can alter the filter condition associated with a configuration table
2017-10-09 03:44:17 +02:00
by calling <function>pg_extension_config_dump</function> again. (This would
2012-12-20 22:30:59 +01:00
typically be useful in an extension update script.) The only way to mark
a table as no longer a configuration table is to dissociate it from the
2017-10-09 03:44:17 +02:00
extension with <command>ALTER EXTENSION ... DROP TABLE</command>.
2012-12-20 22:30:59 +01:00
</para>
2015-03-02 20:12:21 +01:00
<para>
Note that foreign key relationships between these tables will dictate the
order in which the tables are dumped out by pg_dump. Specifically, pg_dump
will attempt to dump the referenced-by table before the referencing table.
As the foreign key relationships are set up at CREATE EXTENSION time (prior
to data being loaded into the tables) circular dependencies are not
supported. When circular dependencies exist, the data will still be dumped
out but the dump will not be able to be restored directly and user
intervention will be required.
</para>
2016-09-07 19:36:08 +02:00
2016-04-18 05:13:45 +02:00
<para>
2017-10-09 03:44:17 +02:00
Sequences associated with <type>serial</type> or <type>bigserial</type> columns
2016-04-18 05:13:45 +02:00
need to be directly marked to dump their state. Marking their parent
relation is not enough for this purpose.
</para>
2011-02-08 22:08:41 +01:00
</sect2>
2023-01-09 21:08:24 +01:00
<sect2 id="extend-extensions-updates">
2011-02-12 03:25:20 +01:00
<title>Extension Updates</title>
<para>
One advantage of the extension mechanism is that it provides convenient
ways to manage updates to the SQL commands that define an extension's
objects. This is done by associating a version name or number with
each released version of the extension's installation script.
In addition, if you want users to be able to update their databases
dynamically from one version to the next, you should provide
2017-10-09 03:44:17 +02:00
<firstterm>update scripts</firstterm> that make the necessary changes to go from
2011-02-12 03:25:20 +01:00
one version to the next. Update scripts have names following the pattern
2019-10-16 06:09:52 +02:00
<literal><replaceable>extension</replaceable>--<replaceable>old_version</replaceable>--<replaceable>target_version</replaceable>.sql</literal>
2017-10-09 03:44:17 +02:00
(for example, <literal>foo--1.0--1.1.sql</literal> contains the commands to modify
version <literal>1.0</literal> of extension <literal>foo</literal> into version
<literal>1.1</literal>).
2011-02-12 03:25:20 +01:00
</para>
<para>
Given that a suitable update script is available, the command
2017-10-09 03:44:17 +02:00
<command>ALTER EXTENSION UPDATE</command> will update an installed extension
2011-02-12 03:25:20 +01:00
to the specified new version. The update script is run in the same
2017-10-09 03:44:17 +02:00
environment that <command>CREATE EXTENSION</command> provides for installation
scripts: in particular, <varname>search_path</varname> is set up in the same
2011-02-12 03:25:20 +01:00
way, and any new objects created by the script are automatically added
Delete deleteWhatDependsOn() in favor of more performDeletion() flag bits.
deleteWhatDependsOn() had grown an uncomfortably large number of
assumptions about what it's used for. There are actually only two minor
differences between what it does and what a regular performDeletion() call
can do, so let's invent additional bits in performDeletion's existing flags
argument that specify those behaviors, and get rid of deleteWhatDependsOn()
as such. (We'd probably have done it this way from the start, except that
performDeletion didn't originally have a flags argument, IIRC.)
Also, add a SKIP_EXTENSIONS flag bit that prevents ever recursing to an
extension, and use that when dropping temporary objects at session end.
This provides a more general solution to the problem addressed in a hacky
way in commit 08dd23cec: if an extension script creates temp objects and
forgets to remove them again, the whole extension went away when its
contained temp objects were deleted. The previous solution only covered
temp relations, but this solves it for all object types.
These changes require minor additions in dependency.c to pass the flags
to subroutines that previously didn't get them, but it's still a net
savings of code, and it seems cleaner than before.
Having done this, revert the special-case code added in 08dd23cec that
prevented addition of pg_depend records for temp table extension
membership, because that caused its own oddities: dropping an extension
that had created such a table didn't automatically remove the table,
leading to a failure if the table had another dependency on the extension
(such as use of an extension data type), or to a duplicate-name failure if
you then tried to recreate the extension. But we keep the part that
prevents the pg_temp_nnn schema from becoming an extension member; we never
want that to happen. Add a regression test case covering these behaviors.
Although this fixes some arguable bugs, we've heard few field complaints,
and any such problems are easily worked around by explicitly dropping temp
objects at the end of extension scripts (which seems like good practice
anyway). So I won't risk a back-patch.
Discussion: https://postgr.es/m/e51f4311-f483-4dd0-1ccc-abec3c405110@BlueTreble.com
2016-12-02 20:57:35 +01:00
to the extension. Also, if the script chooses to drop extension member
objects, they are automatically dissociated from the extension.
2011-02-12 03:25:20 +01:00
</para>
2011-02-12 22:40:41 +01:00
<para>
If an extension has secondary control files, the control parameters
that are used for an update script are those associated with the script's
target (new) version.
</para>
2011-02-12 03:25:20 +01:00
<para>
2017-10-09 03:44:17 +02:00
<command>ALTER EXTENSION</command> is able to execute sequences of update
2011-02-12 03:25:20 +01:00
script files to achieve a requested update. For example, if only
2017-10-09 03:44:17 +02:00
<literal>foo--1.0--1.1.sql</literal> and <literal>foo--1.1--2.0.sql</literal> are
available, <command>ALTER EXTENSION</command> will apply them in sequence if an
update to version <literal>2.0</literal> is requested when <literal>1.0</literal> is
2011-02-12 03:25:20 +01:00
currently installed.
</para>
<para>
2017-10-09 03:44:17 +02:00
<productname>PostgreSQL</productname> doesn't assume anything about the properties
of version names: for example, it does not know whether <literal>1.1</literal>
follows <literal>1.0</literal>. It just matches up the available version names
2011-02-12 03:25:20 +01:00
and follows the path that requires applying the fewest update scripts.
2011-02-13 19:03:41 +01:00
(A version name can actually be any string that doesn't contain
2017-10-09 03:44:17 +02:00
<literal>--</literal> or leading or trailing <literal>-</literal>.)
2011-02-12 03:25:20 +01:00
</para>
<para>
2017-10-09 03:44:17 +02:00
Sometimes it is useful to provide <quote>downgrade</quote> scripts, for
example <literal>foo--1.1--1.0.sql</literal> to allow reverting the changes
associated with version <literal>1.1</literal>. If you do that, be careful
2011-02-12 03:25:20 +01:00
of the possibility that a downgrade script might unexpectedly
get applied because it yields a shorter path. The risky case is where
2017-10-09 03:44:17 +02:00
there is a <quote>fast path</quote> update script that jumps ahead several
2011-02-12 03:25:20 +01:00
versions as well as a downgrade script to the fast path's start point.
It might take fewer steps to apply the downgrade and then the fast
path than to move ahead one version at a time. If the downgrade script
drops any irreplaceable objects, this will yield undesirable results.
</para>
2011-02-14 22:07:00 +01:00
<para>
To check for unexpected update paths, use this command:
<programlisting>
2017-10-09 03:44:17 +02:00
SELECT * FROM pg_extension_update_paths('<replaceable>extension_name</replaceable>');
2011-02-14 22:07:00 +01:00
</programlisting>
This shows each pair of distinct known version names for the specified
extension, together with the update path sequence that would be taken to
2017-10-09 03:44:17 +02:00
get from the source version to the target version, or <literal>NULL</literal> if
2011-02-14 22:07:00 +01:00
there is no available update path. The path is shown in textual form
2017-10-09 03:44:17 +02:00
with <literal>--</literal> separators. You can use
<literal>regexp_split_to_array(path,'--')</literal> if you prefer an array
2011-02-14 22:07:00 +01:00
format.
</para>
2011-02-12 03:25:20 +01:00
</sect2>
2023-01-09 21:08:24 +01:00
<sect2 id="extend-extensions-update-scripts">
2019-09-08 10:26:35 +02:00
<title>Installing Extensions Using Update Scripts</title>
Allow CREATE EXTENSION to follow extension update paths.
Previously, to update an extension you had to produce both a version-update
script and a new base installation script. It's become more and more
obvious that that's tedious, duplicative, and error-prone. This patch
attempts to improve matters by allowing the new base installation script
to be omitted. CREATE EXTENSION will install a requested version if it
can find a base script and a chain of update scripts that will get there.
As in the existing update logic, shorter chains are preferred if there's
more than one possibility, with an arbitrary tie-break rule for chains
of equal length.
Also adjust the pg_available_extension_versions view to show such versions
as installable.
While at it, refactor the code so that CASCADE processing works for
extensions requested during ApplyExtensionUpdates(). Without this,
addition of a new requirement in an updated extension would require
creating a new base script, even if there was no other reason to do that.
(It would be easy at this point to add a CASCADE option to ALTER EXTENSION
UPDATE, to allow the same thing to happen during a manually-commanded
version update, but I have not done that here.)
Tom Lane, reviewed by Andres Freund
Discussion: <20160905005919.jz2m2yh3und2dsuy@alap3.anarazel.de>
2016-09-11 20:15:07 +02:00
<para>
An extension that has been around for awhile will probably exist in
several versions, for which the author will need to write update scripts.
2017-10-09 03:44:17 +02:00
For example, if you have released a <literal>foo</literal> extension in
versions <literal>1.0</literal>, <literal>1.1</literal>, and <literal>1.2</literal>, there
should be update scripts <filename>foo--1.0--1.1.sql</filename>
and <filename>foo--1.1--1.2.sql</filename>.
Before <productname>PostgreSQL</productname> 10, it was necessary to also create
new script files <filename>foo--1.1.sql</filename> and <filename>foo--1.2.sql</filename>
Allow CREATE EXTENSION to follow extension update paths.
Previously, to update an extension you had to produce both a version-update
script and a new base installation script. It's become more and more
obvious that that's tedious, duplicative, and error-prone. This patch
attempts to improve matters by allowing the new base installation script
to be omitted. CREATE EXTENSION will install a requested version if it
can find a base script and a chain of update scripts that will get there.
As in the existing update logic, shorter chains are preferred if there's
more than one possibility, with an arbitrary tie-break rule for chains
of equal length.
Also adjust the pg_available_extension_versions view to show such versions
as installable.
While at it, refactor the code so that CASCADE processing works for
extensions requested during ApplyExtensionUpdates(). Without this,
addition of a new requirement in an updated extension would require
creating a new base script, even if there was no other reason to do that.
(It would be easy at this point to add a CASCADE option to ALTER EXTENSION
UPDATE, to allow the same thing to happen during a manually-commanded
version update, but I have not done that here.)
Tom Lane, reviewed by Andres Freund
Discussion: <20160905005919.jz2m2yh3und2dsuy@alap3.anarazel.de>
2016-09-11 20:15:07 +02:00
that directly build the newer extension versions, or else the newer
versions could not be installed directly, only by
2017-10-09 03:44:17 +02:00
installing <literal>1.0</literal> and then updating. That was tedious and
Allow CREATE EXTENSION to follow extension update paths.
Previously, to update an extension you had to produce both a version-update
script and a new base installation script. It's become more and more
obvious that that's tedious, duplicative, and error-prone. This patch
attempts to improve matters by allowing the new base installation script
to be omitted. CREATE EXTENSION will install a requested version if it
can find a base script and a chain of update scripts that will get there.
As in the existing update logic, shorter chains are preferred if there's
more than one possibility, with an arbitrary tie-break rule for chains
of equal length.
Also adjust the pg_available_extension_versions view to show such versions
as installable.
While at it, refactor the code so that CASCADE processing works for
extensions requested during ApplyExtensionUpdates(). Without this,
addition of a new requirement in an updated extension would require
creating a new base script, even if there was no other reason to do that.
(It would be easy at this point to add a CASCADE option to ALTER EXTENSION
UPDATE, to allow the same thing to happen during a manually-commanded
version update, but I have not done that here.)
Tom Lane, reviewed by Andres Freund
Discussion: <20160905005919.jz2m2yh3und2dsuy@alap3.anarazel.de>
2016-09-11 20:15:07 +02:00
duplicative, but now it's unnecessary, because <command>CREATE
2017-10-09 03:44:17 +02:00
EXTENSION</command> can follow update chains automatically.
Allow CREATE EXTENSION to follow extension update paths.
Previously, to update an extension you had to produce both a version-update
script and a new base installation script. It's become more and more
obvious that that's tedious, duplicative, and error-prone. This patch
attempts to improve matters by allowing the new base installation script
to be omitted. CREATE EXTENSION will install a requested version if it
can find a base script and a chain of update scripts that will get there.
As in the existing update logic, shorter chains are preferred if there's
more than one possibility, with an arbitrary tie-break rule for chains
of equal length.
Also adjust the pg_available_extension_versions view to show such versions
as installable.
While at it, refactor the code so that CASCADE processing works for
extensions requested during ApplyExtensionUpdates(). Without this,
addition of a new requirement in an updated extension would require
creating a new base script, even if there was no other reason to do that.
(It would be easy at this point to add a CASCADE option to ALTER EXTENSION
UPDATE, to allow the same thing to happen during a manually-commanded
version update, but I have not done that here.)
Tom Lane, reviewed by Andres Freund
Discussion: <20160905005919.jz2m2yh3und2dsuy@alap3.anarazel.de>
2016-09-11 20:15:07 +02:00
For example, if only the script
2017-10-09 03:44:17 +02:00
files <filename>foo--1.0.sql</filename>, <filename>foo--1.0--1.1.sql</filename>,
and <filename>foo--1.1--1.2.sql</filename> are available then a request to
install version <literal>1.2</literal> is honored by running those three
Allow CREATE EXTENSION to follow extension update paths.
Previously, to update an extension you had to produce both a version-update
script and a new base installation script. It's become more and more
obvious that that's tedious, duplicative, and error-prone. This patch
attempts to improve matters by allowing the new base installation script
to be omitted. CREATE EXTENSION will install a requested version if it
can find a base script and a chain of update scripts that will get there.
As in the existing update logic, shorter chains are preferred if there's
more than one possibility, with an arbitrary tie-break rule for chains
of equal length.
Also adjust the pg_available_extension_versions view to show such versions
as installable.
While at it, refactor the code so that CASCADE processing works for
extensions requested during ApplyExtensionUpdates(). Without this,
addition of a new requirement in an updated extension would require
creating a new base script, even if there was no other reason to do that.
(It would be easy at this point to add a CASCADE option to ALTER EXTENSION
UPDATE, to allow the same thing to happen during a manually-commanded
version update, but I have not done that here.)
Tom Lane, reviewed by Andres Freund
Discussion: <20160905005919.jz2m2yh3und2dsuy@alap3.anarazel.de>
2016-09-11 20:15:07 +02:00
scripts in sequence. The processing is the same as if you'd first
2017-10-09 03:44:17 +02:00
installed <literal>1.0</literal> and then updated to <literal>1.2</literal>.
(As with <command>ALTER EXTENSION UPDATE</command>, if multiple pathways are
Allow CREATE EXTENSION to follow extension update paths.
Previously, to update an extension you had to produce both a version-update
script and a new base installation script. It's become more and more
obvious that that's tedious, duplicative, and error-prone. This patch
attempts to improve matters by allowing the new base installation script
to be omitted. CREATE EXTENSION will install a requested version if it
can find a base script and a chain of update scripts that will get there.
As in the existing update logic, shorter chains are preferred if there's
more than one possibility, with an arbitrary tie-break rule for chains
of equal length.
Also adjust the pg_available_extension_versions view to show such versions
as installable.
While at it, refactor the code so that CASCADE processing works for
extensions requested during ApplyExtensionUpdates(). Without this,
addition of a new requirement in an updated extension would require
creating a new base script, even if there was no other reason to do that.
(It would be easy at this point to add a CASCADE option to ALTER EXTENSION
UPDATE, to allow the same thing to happen during a manually-commanded
version update, but I have not done that here.)
Tom Lane, reviewed by Andres Freund
Discussion: <20160905005919.jz2m2yh3und2dsuy@alap3.anarazel.de>
2016-09-11 20:15:07 +02:00
available then the shortest is preferred.) Arranging an extension's
script files in this style can reduce the amount of maintenance effort
needed to produce small updates.
</para>
<para>
If you use secondary (version-specific) control files with an extension
maintained in this style, keep in mind that each version needs a control
file even if it has no stand-alone installation script, as that control
file will determine how the implicit update to that version is performed.
2017-10-09 03:44:17 +02:00
For example, if <filename>foo--1.0.control</filename> specifies <literal>requires
= 'bar'</literal> but <literal>foo</literal>'s other control files do not, the
extension's dependency on <literal>bar</literal> will be dropped when updating
from <literal>1.0</literal> to another version.
Allow CREATE EXTENSION to follow extension update paths.
Previously, to update an extension you had to produce both a version-update
script and a new base installation script. It's become more and more
obvious that that's tedious, duplicative, and error-prone. This patch
attempts to improve matters by allowing the new base installation script
to be omitted. CREATE EXTENSION will install a requested version if it
can find a base script and a chain of update scripts that will get there.
As in the existing update logic, shorter chains are preferred if there's
more than one possibility, with an arbitrary tie-break rule for chains
of equal length.
Also adjust the pg_available_extension_versions view to show such versions
as installable.
While at it, refactor the code so that CASCADE processing works for
extensions requested during ApplyExtensionUpdates(). Without this,
addition of a new requirement in an updated extension would require
creating a new base script, even if there was no other reason to do that.
(It would be easy at this point to add a CASCADE option to ALTER EXTENSION
UPDATE, to allow the same thing to happen during a manually-commanded
version update, but I have not done that here.)
Tom Lane, reviewed by Andres Freund
Discussion: <20160905005919.jz2m2yh3und2dsuy@alap3.anarazel.de>
2016-09-11 20:15:07 +02:00
</para>
</sect2>
Make contrib modules' installation scripts more secure.
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
2020-08-10 16:44:42 +02:00
<sect2 id="extend-extensions-security">
<title>Security Considerations for Extensions</title>
<para>
Widely-distributed extensions should assume little about the database
they occupy. Therefore, it's appropriate to write functions provided
by an extension in a secure style that cannot be compromised by
search-path-based attacks.
</para>
<para>
An extension that has the <varname>superuser</varname> property set to
true must also consider security hazards for the actions taken within
its installation and update scripts. It is not terribly difficult for
a malicious user to create trojan-horse objects that will compromise
later execution of a carelessly-written extension script, allowing that
user to acquire superuser privileges.
</para>
<para>
If an extension is marked <varname>trusted</varname>, then its
installation schema can be selected by the installing user, who might
intentionally use an insecure schema in hopes of gaining superuser
privileges. Therefore, a trusted extension is extremely exposed from a
security standpoint, and all its script commands must be carefully
examined to ensure that no compromise is possible.
</para>
<para>
Advice about writing functions securely is provided in
<xref linkend="extend-extensions-security-funcs"/> below, and advice
about writing installation scripts securely is provided in
<xref linkend="extend-extensions-security-scripts"/>.
</para>
<sect3 id="extend-extensions-security-funcs">
<title>Security Considerations for Extension Functions</title>
<para>
SQL-language and PL-language functions provided by extensions are at
risk of search-path-based attacks when they are executed, since
parsing of these functions occurs at execution time not creation time.
</para>
<para>
The <link linkend="sql-createfunction-security"><command>CREATE
FUNCTION</command></link> reference page contains advice about
writing <literal>SECURITY DEFINER</literal> functions safely. It's
good practice to apply those techniques for any function provided by
an extension, since the function might be called by a high-privilege
user.
</para>
<!-- XXX It's not enough to use qualified names, because one might write a
qualified name to an object that itself uses unqualified names. Many
information_schema functions have that defect, for example. However,
that's a defect in the referenced object, and relatively few queries
will be affected. Also, we direct applications to secure search_path
when connecting to an untrusted database; if applications do that,
they are immune to known attacks even if some extension refers to a
defective object. Therefore, guide extension authors as though core
PostgreSQL contained no such defect. -->
<para>
If you cannot set the <varname>search_path</varname> to contain only
secure schemas, assume that each unqualified name could resolve to an
object that a malicious user has defined. Beware of constructs that
depend on <varname>search_path</varname> implicitly; for
example, <token>IN</token>
and <literal>CASE <replaceable>expression</replaceable> WHEN</literal>
always select an operator using the search path. In their place, use
<literal>OPERATOR(<replaceable>schema</replaceable>.=) ANY</literal>
and <literal>CASE WHEN <replaceable>expression</replaceable></literal>.
</para>
<para>
A general-purpose extension usually should not assume that it's been
installed into a secure schema, which means that even schema-qualified
references to its own objects are not entirely risk-free. For
example, if the extension has defined a
function <literal>myschema.myfunc(bigint)</literal> then a call such
as <literal>myschema.myfunc(42)</literal> could be captured by a
hostile function <literal>myschema.myfunc(integer)</literal>. Be
careful that the data types of function and operator parameters exactly
match the declared argument types, using explicit casts where necessary.
</para>
</sect3>
<sect3 id="extend-extensions-security-scripts">
<title>Security Considerations for Extension Scripts</title>
<para>
An extension installation or update script should be written to guard
against search-path-based attacks occurring when the script executes.
If an object reference in the script can be made to resolve to some
other object than the script author intended, then a compromise might
occur immediately, or later when the mis-defined extension object is
used.
</para>
<para>
DDL commands such as <command>CREATE FUNCTION</command>
and <command>CREATE OPERATOR CLASS</command> are generally secure,
but beware of any command having a general-purpose expression as a
component. For example, <command>CREATE VIEW</command> needs to be
vetted, as does a <literal>DEFAULT</literal> expression
in <command>CREATE FUNCTION</command>.
</para>
<para>
Sometimes an extension script might need to execute general-purpose
SQL, for example to make catalog adjustments that aren't possible via
DDL. Be careful to execute such commands with a
secure <varname>search_path</varname>; do <emphasis>not</emphasis>
trust the path provided by <command>CREATE/ALTER EXTENSION</command>
to be secure. Best practice is to temporarily
set <varname>search_path</varname> to <literal>'pg_catalog,
pg_temp'</literal> and insert references to the extension's
installation schema explicitly where needed. (This practice might
also be helpful for creating views.) Examples can be found in
the <filename>contrib</filename> modules in
the <productname>PostgreSQL</productname> source code distribution.
</para>
<para>
Cross-extension references are extremely difficult to make fully
secure, partially because of uncertainty about which schema the other
extension is in. The hazards are reduced if both extensions are
installed in the same schema, because then a hostile object cannot be
placed ahead of the referenced extension in the installation-time
<varname>search_path</varname>. However, no mechanism currently exists
to require that. For now, best practice is to not mark an extension
trusted if it depends on another one, unless that other one is always
installed in <literal>pg_catalog</literal>.
</para>
</sect3>
</sect2>
2016-09-07 19:36:08 +02:00
<sect2 id="extend-extensions-example">
2011-02-08 22:08:41 +01:00
<title>Extension Example</title>
<para>
2017-10-09 03:44:17 +02:00
Here is a complete example of an <acronym>SQL</acronym>-only
2011-02-08 22:08:41 +01:00
extension, a two-element composite type that can store any type of value
2017-10-09 03:44:17 +02:00
in its slots, which are named <quote>k</quote> and <quote>v</quote>. Non-text
2011-02-08 22:08:41 +01:00
values are automatically coerced to text for storage.
</para>
<para>
2017-10-09 03:44:17 +02:00
The script file <filename>pair--1.0.sql</filename> looks like this:
2011-02-08 22:08:41 +01:00
<programlisting><![CDATA[
2011-10-12 21:45:03 +02:00
-- complain if script is sourced in psql, rather than via CREATE EXTENSION
\echo Use "CREATE EXTENSION pair" to load this file. \quit
2018-03-24 04:31:03 +01:00
CREATE TYPE pair AS ( k text, v text );
2011-02-08 22:08:41 +01:00
Make contrib modules' installation scripts more secure.
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
2020-08-10 16:44:42 +02:00
CREATE FUNCTION pair(text, text)
Document security implications of search_path and the public schema.
The ability to create like-named objects in different schemas opens up
the potential for users to change the behavior of other users' queries,
maliciously or accidentally. When you connect to a PostgreSQL server,
you should remove from your search_path any schema for which a user
other than yourself or superusers holds the CREATE privilege. If you do
not, other users holding CREATE privilege can redefine the behavior of
your commands, causing them to perform arbitrary SQL statements under
your identity. "SET search_path = ..." and "SELECT
pg_catalog.set_config(...)" are not vulnerable to such hijacking, so one
can use either as the first command of a session. As special
exceptions, the following client applications behave as documented
regardless of search_path settings and schema privileges: clusterdb
createdb createlang createuser dropdb droplang dropuser ecpg (not
programs it generates) initdb oid2name pg_archivecleanup pg_basebackup
pg_config pg_controldata pg_ctl pg_dump pg_dumpall pg_isready
pg_receivewal pg_recvlogical pg_resetwal pg_restore pg_rewind pg_standby
pg_test_fsync pg_test_timing pg_upgrade pg_waldump reindexdb vacuumdb
vacuumlo. Not included are core client programs that run user-specified
SQL commands, namely psql and pgbench. PostgreSQL encourages non-core
client applications to do likewise.
Document this in the context of libpq connections, psql connections,
dblink connections, ECPG connections, extension packaging, and schema
usage patterns. The principal defense for applications is "SELECT
pg_catalog.set_config('search_path', '', false)", and the principal
defense for databases is "REVOKE CREATE ON SCHEMA public FROM PUBLIC".
Either one is sufficient to prevent attack. After a REVOKE, consider
auditing the public schema for objects named like pg_catalog objects.
Authors of SECURITY DEFINER functions use some of the same defenses, and
the CREATE FUNCTION reference page already covered them thoroughly.
This is a good opportunity to audit SECURITY DEFINER functions for
robust security practice.
Back-patch to 9.3 (all supported versions).
Reviewed by Michael Paquier and Jonathan S. Katz. Reported by Arseniy
Sharoglazov.
Security: CVE-2018-1058
2018-02-26 16:39:44 +01:00
RETURNS pair LANGUAGE SQL AS 'SELECT ROW($1, $2)::@extschema@.pair;';
2011-02-08 22:08:41 +01:00
2018-08-15 18:05:46 +02:00
CREATE OPERATOR ~> (LEFTARG = text, RIGHTARG = text, FUNCTION = pair);
2011-02-08 22:08:41 +01:00
Document security implications of search_path and the public schema.
The ability to create like-named objects in different schemas opens up
the potential for users to change the behavior of other users' queries,
maliciously or accidentally. When you connect to a PostgreSQL server,
you should remove from your search_path any schema for which a user
other than yourself or superusers holds the CREATE privilege. If you do
not, other users holding CREATE privilege can redefine the behavior of
your commands, causing them to perform arbitrary SQL statements under
your identity. "SET search_path = ..." and "SELECT
pg_catalog.set_config(...)" are not vulnerable to such hijacking, so one
can use either as the first command of a session. As special
exceptions, the following client applications behave as documented
regardless of search_path settings and schema privileges: clusterdb
createdb createlang createuser dropdb droplang dropuser ecpg (not
programs it generates) initdb oid2name pg_archivecleanup pg_basebackup
pg_config pg_controldata pg_ctl pg_dump pg_dumpall pg_isready
pg_receivewal pg_recvlogical pg_resetwal pg_restore pg_rewind pg_standby
pg_test_fsync pg_test_timing pg_upgrade pg_waldump reindexdb vacuumdb
vacuumlo. Not included are core client programs that run user-specified
SQL commands, namely psql and pgbench. PostgreSQL encourages non-core
client applications to do likewise.
Document this in the context of libpq connections, psql connections,
dblink connections, ECPG connections, extension packaging, and schema
usage patterns. The principal defense for applications is "SELECT
pg_catalog.set_config('search_path', '', false)", and the principal
defense for databases is "REVOKE CREATE ON SCHEMA public FROM PUBLIC".
Either one is sufficient to prevent attack. After a REVOKE, consider
auditing the public schema for objects named like pg_catalog objects.
Authors of SECURITY DEFINER functions use some of the same defenses, and
the CREATE FUNCTION reference page already covered them thoroughly.
This is a good opportunity to audit SECURITY DEFINER functions for
robust security practice.
Back-patch to 9.3 (all supported versions).
Reviewed by Michael Paquier and Jonathan S. Katz. Reported by Arseniy
Sharoglazov.
Security: CVE-2018-1058
2018-02-26 16:39:44 +01:00
-- "SET search_path" is easy to get right, but qualified names perform better.
Make contrib modules' installation scripts more secure.
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
2020-08-10 16:44:42 +02:00
CREATE FUNCTION lower(pair)
Document security implications of search_path and the public schema.
The ability to create like-named objects in different schemas opens up
the potential for users to change the behavior of other users' queries,
maliciously or accidentally. When you connect to a PostgreSQL server,
you should remove from your search_path any schema for which a user
other than yourself or superusers holds the CREATE privilege. If you do
not, other users holding CREATE privilege can redefine the behavior of
your commands, causing them to perform arbitrary SQL statements under
your identity. "SET search_path = ..." and "SELECT
pg_catalog.set_config(...)" are not vulnerable to such hijacking, so one
can use either as the first command of a session. As special
exceptions, the following client applications behave as documented
regardless of search_path settings and schema privileges: clusterdb
createdb createlang createuser dropdb droplang dropuser ecpg (not
programs it generates) initdb oid2name pg_archivecleanup pg_basebackup
pg_config pg_controldata pg_ctl pg_dump pg_dumpall pg_isready
pg_receivewal pg_recvlogical pg_resetwal pg_restore pg_rewind pg_standby
pg_test_fsync pg_test_timing pg_upgrade pg_waldump reindexdb vacuumdb
vacuumlo. Not included are core client programs that run user-specified
SQL commands, namely psql and pgbench. PostgreSQL encourages non-core
client applications to do likewise.
Document this in the context of libpq connections, psql connections,
dblink connections, ECPG connections, extension packaging, and schema
usage patterns. The principal defense for applications is "SELECT
pg_catalog.set_config('search_path', '', false)", and the principal
defense for databases is "REVOKE CREATE ON SCHEMA public FROM PUBLIC".
Either one is sufficient to prevent attack. After a REVOKE, consider
auditing the public schema for objects named like pg_catalog objects.
Authors of SECURITY DEFINER functions use some of the same defenses, and
the CREATE FUNCTION reference page already covered them thoroughly.
This is a good opportunity to audit SECURITY DEFINER functions for
robust security practice.
Back-patch to 9.3 (all supported versions).
Reviewed by Michael Paquier and Jonathan S. Katz. Reported by Arseniy
Sharoglazov.
Security: CVE-2018-1058
2018-02-26 16:39:44 +01:00
RETURNS pair LANGUAGE SQL
AS 'SELECT ROW(lower($1.k), lower($1.v))::@extschema@.pair;'
SET search_path = pg_temp;
2011-02-08 22:08:41 +01:00
Make contrib modules' installation scripts more secure.
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
2020-08-10 16:44:42 +02:00
CREATE FUNCTION pair_concat(pair, pair)
Document security implications of search_path and the public schema.
The ability to create like-named objects in different schemas opens up
the potential for users to change the behavior of other users' queries,
maliciously or accidentally. When you connect to a PostgreSQL server,
you should remove from your search_path any schema for which a user
other than yourself or superusers holds the CREATE privilege. If you do
not, other users holding CREATE privilege can redefine the behavior of
your commands, causing them to perform arbitrary SQL statements under
your identity. "SET search_path = ..." and "SELECT
pg_catalog.set_config(...)" are not vulnerable to such hijacking, so one
can use either as the first command of a session. As special
exceptions, the following client applications behave as documented
regardless of search_path settings and schema privileges: clusterdb
createdb createlang createuser dropdb droplang dropuser ecpg (not
programs it generates) initdb oid2name pg_archivecleanup pg_basebackup
pg_config pg_controldata pg_ctl pg_dump pg_dumpall pg_isready
pg_receivewal pg_recvlogical pg_resetwal pg_restore pg_rewind pg_standby
pg_test_fsync pg_test_timing pg_upgrade pg_waldump reindexdb vacuumdb
vacuumlo. Not included are core client programs that run user-specified
SQL commands, namely psql and pgbench. PostgreSQL encourages non-core
client applications to do likewise.
Document this in the context of libpq connections, psql connections,
dblink connections, ECPG connections, extension packaging, and schema
usage patterns. The principal defense for applications is "SELECT
pg_catalog.set_config('search_path', '', false)", and the principal
defense for databases is "REVOKE CREATE ON SCHEMA public FROM PUBLIC".
Either one is sufficient to prevent attack. After a REVOKE, consider
auditing the public schema for objects named like pg_catalog objects.
Authors of SECURITY DEFINER functions use some of the same defenses, and
the CREATE FUNCTION reference page already covered them thoroughly.
This is a good opportunity to audit SECURITY DEFINER functions for
robust security practice.
Back-patch to 9.3 (all supported versions).
Reviewed by Michael Paquier and Jonathan S. Katz. Reported by Arseniy
Sharoglazov.
Security: CVE-2018-1058
2018-02-26 16:39:44 +01:00
RETURNS pair LANGUAGE SQL
AS 'SELECT ROW($1.k OPERATOR(pg_catalog.||) $2.k,
$1.v OPERATOR(pg_catalog.||) $2.v)::@extschema@.pair;';
2011-02-08 22:08:41 +01:00
]]>
</programlisting>
</para>
<para>
2017-10-09 03:44:17 +02:00
The control file <filename>pair.control</filename> looks like this:
2011-02-08 22:08:41 +01:00
<programlisting>
# pair extension
comment = 'A key/value pair data type'
2011-02-12 03:25:20 +01:00
default_version = '1.0'
Make contrib modules' installation scripts more secure.
Hostile objects located within the installation-time search_path could
capture references in an extension's installation or upgrade script.
If the extension is being installed with superuser privileges, this
opens the door to privilege escalation. While such hazards have existed
all along, their urgency increases with the v13 "trusted extensions"
feature, because that lets a non-superuser control the installation path
for a superuser-privileged script. Therefore, make a number of changes
to make such situations more secure:
* Tweak the construction of the installation-time search_path to ensure
that references to objects in pg_catalog can't be subverted; and
explicitly add pg_temp to the end of the path to prevent attacks using
temporary objects.
* Disable check_function_bodies within installation/upgrade scripts,
so that any security gaps in SQL-language or PL-language function bodies
cannot create a risk of unwanted installation-time code execution.
* Adjust lookup of type input/receive functions and join estimator
functions to complain if there are multiple candidate functions. This
prevents capture of references to functions whose signature is not the
first one checked; and it's arguably more user-friendly anyway.
* Modify various contrib upgrade scripts to ensure that catalog
modification queries are executed with secure search paths. (These
are in-place modifications with no extension version changes, since
it is the update process itself that is at issue, not the end result.)
Extensions that depend on other extensions cannot be made fully secure
by these methods alone; therefore, revert the "trusted" marking that
commit eb67623c9 applied to earthdistance and hstore_plperl, pending
some better solution to that set of issues.
Also add documentation around these issues, to help extension authors
write secure installation scripts.
Patch by me, following an observation by Andres Freund; thanks
to Noah Misch for review.
Security: CVE-2020-14350
2020-08-10 16:44:42 +02:00
# cannot be relocatable because of use of @extschema@
Document security implications of search_path and the public schema.
The ability to create like-named objects in different schemas opens up
the potential for users to change the behavior of other users' queries,
maliciously or accidentally. When you connect to a PostgreSQL server,
you should remove from your search_path any schema for which a user
other than yourself or superusers holds the CREATE privilege. If you do
not, other users holding CREATE privilege can redefine the behavior of
your commands, causing them to perform arbitrary SQL statements under
your identity. "SET search_path = ..." and "SELECT
pg_catalog.set_config(...)" are not vulnerable to such hijacking, so one
can use either as the first command of a session. As special
exceptions, the following client applications behave as documented
regardless of search_path settings and schema privileges: clusterdb
createdb createlang createuser dropdb droplang dropuser ecpg (not
programs it generates) initdb oid2name pg_archivecleanup pg_basebackup
pg_config pg_controldata pg_ctl pg_dump pg_dumpall pg_isready
pg_receivewal pg_recvlogical pg_resetwal pg_restore pg_rewind pg_standby
pg_test_fsync pg_test_timing pg_upgrade pg_waldump reindexdb vacuumdb
vacuumlo. Not included are core client programs that run user-specified
SQL commands, namely psql and pgbench. PostgreSQL encourages non-core
client applications to do likewise.
Document this in the context of libpq connections, psql connections,
dblink connections, ECPG connections, extension packaging, and schema
usage patterns. The principal defense for applications is "SELECT
pg_catalog.set_config('search_path', '', false)", and the principal
defense for databases is "REVOKE CREATE ON SCHEMA public FROM PUBLIC".
Either one is sufficient to prevent attack. After a REVOKE, consider
auditing the public schema for objects named like pg_catalog objects.
Authors of SECURITY DEFINER functions use some of the same defenses, and
the CREATE FUNCTION reference page already covered them thoroughly.
This is a good opportunity to audit SECURITY DEFINER functions for
robust security practice.
Back-patch to 9.3 (all supported versions).
Reviewed by Michael Paquier and Jonathan S. Katz. Reported by Arseniy
Sharoglazov.
Security: CVE-2018-1058
2018-02-26 16:39:44 +01:00
relocatable = false
2011-02-08 22:08:41 +01:00
</programlisting>
</para>
<para>
While you hardly need a makefile to install these two files into the
2017-10-09 03:44:17 +02:00
correct directory, you could use a <filename>Makefile</filename> containing this:
2011-02-08 22:08:41 +01:00
<programlisting>
EXTENSION = pair
2011-02-13 19:03:41 +01:00
DATA = pair--1.0.sql
2011-02-08 22:08:41 +01:00
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
</programlisting>
This makefile relies on <acronym>PGXS</acronym>, which is described
2017-11-23 15:39:47 +01:00
in <xref linkend="extend-pgxs"/>. The command <literal>make install</literal>
2011-02-12 22:40:41 +01:00
will install the control and script files into the correct
2017-10-09 03:44:17 +02:00
directory as reported by <application>pg_config</application>.
2011-02-08 22:08:41 +01:00
</para>
<para>
Once the files are installed, use the
Improve <xref> vs. <command> formatting in the documentation
SQL commands are generally marked up as <command>, except when a link
to a reference page is used using <xref>. But the latter doesn't
create monospace markup, so this looks strange especially when a
paragraph contains a mix of links and non-links.
We considered putting <command> in the <refentrytitle> on the target
side, but that creates some formatting side effects elsewhere.
Generally, it seems safer to solve this on the link source side.
We can't put the <xref> inside the <command>; the DTD doesn't allow
this. DocBook 5 would allow the <command> to have the linkend
attribute itself, but we are not there yet.
So to solve this for now, convert the <xref>s to <link> plus
<command>. This gives the correct look and also gives some more
flexibility what we can put into the link text (e.g., subcommands or
other clauses). In the future, these could then be converted to
DocBook 5 style.
I haven't converted absolutely all xrefs to SQL command reference
pages, only those where we care about the appearance of the link text
or where it was otherwise appropriate to make the appearance match a
bit better. Also in some cases, the links where repetitive, so in
those cases the links where just removed and replaced by a plain
<command>. In cases where we just want the link and don't
specifically care about the generated link text (typically phrased
"for further information see <xref ...>") the xref is kept.
Reported-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
Discussion: https://www.postgresql.org/message-id/flat/87o8pco34z.fsf@wibble.ilmari.org
2020-10-03 16:16:51 +02:00
<command>CREATE EXTENSION</command> command to load the objects into
2011-02-08 22:08:41 +01:00
any particular database.
</para>
</sect2>
</sect1>
<sect1 id="extend-pgxs">
<title>Extension Building Infrastructure</title>
<indexterm zone="extend-pgxs">
<primary>pgxs</primary>
</indexterm>
<para>
If you are thinking about distributing your
2017-10-09 03:44:17 +02:00
<productname>PostgreSQL</productname> extension modules, setting up a
2011-02-08 22:08:41 +01:00
portable build system for them can be fairly difficult. Therefore
2017-10-09 03:44:17 +02:00
the <productname>PostgreSQL</productname> installation provides a build
2011-02-08 22:08:41 +01:00
infrastructure for extensions, called <acronym>PGXS</acronym>, so
that simple extension modules can be built simply against an
already installed server. <acronym>PGXS</acronym> is mainly intended
for extensions that include C code, although it can be used for
pure-SQL extensions too. Note that <acronym>PGXS</acronym> is not
intended to be a universal build system framework that can be used
2017-10-09 03:44:17 +02:00
to build any software interfacing to <productname>PostgreSQL</productname>;
2011-02-08 22:08:41 +01:00
it simply automates common build rules for simple server extension
modules. For more complicated packages, you might need to write your
own build system.
2010-06-01 04:31:36 +02:00
</para>
2010-06-03 16:39:58 +02:00
<para>
2011-02-08 22:08:41 +01:00
To use the <acronym>PGXS</acronym> infrastructure for your extension,
you must write a simple makefile.
In the makefile, you need to set some variables
2013-10-07 04:57:16 +02:00
and include the global <acronym>PGXS</acronym> makefile.
2011-02-08 22:08:41 +01:00
Here is an example that builds an extension module named
<literal>isbn_issn</literal>, consisting of a shared library containing
2021-06-11 03:38:04 +02:00
some C code, an extension control file, an SQL script, an include file
2018-07-31 20:58:39 +02:00
(only needed if other modules might need to access the extension functions
without going via SQL), and a documentation text file:
2011-02-08 22:08:41 +01:00
<programlisting>
MODULES = isbn_issn
EXTENSION = isbn_issn
2011-02-13 19:03:41 +01:00
DATA = isbn_issn--1.0.sql
2011-02-08 22:08:41 +01:00
DOCS = README.isbn_issn
2018-07-31 20:58:39 +02:00
HEADERS_isbn_issn = isbn_issn.h
2011-02-08 22:08:41 +01:00
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
</programlisting>
The last three lines should always be the same. Earlier in the
file, you assign variables or add custom
<application>make</application> rules.
2010-06-03 16:39:58 +02:00
</para>
2011-02-08 22:08:41 +01:00
<para>
Set one of these three variables to specify what is built:
<variablelist>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-modules">
2011-02-08 22:08:41 +01:00
<term><varname>MODULES</varname></term>
<listitem>
<para>
list of shared-library objects to be built from source files with same
stem (do not include library suffixes in this list)
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-module-big">
2011-02-08 22:08:41 +01:00
<term><varname>MODULE_big</varname></term>
<listitem>
<para>
a shared library to build from multiple source files
(list object files in <varname>OBJS</varname>)
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-program">
2011-02-08 22:08:41 +01:00
<term><varname>PROGRAM</varname></term>
<listitem>
<para>
an executable program to build
(list object files in <varname>OBJS</varname>)
</para>
</listitem>
</varlistentry>
</variablelist>
The following variables can also be set:
<variablelist>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-extension">
2011-02-12 04:53:43 +01:00
<term><varname>EXTENSION</varname></term>
2011-02-08 22:08:41 +01:00
<listitem>
<para>
2011-02-12 04:53:43 +01:00
extension name(s); for each name you must provide an
<literal><replaceable>extension</replaceable>.control</literal> file,
which will be installed into
<literal><replaceable>prefix</replaceable>/share/extension</literal>
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-moduledir">
2011-02-12 04:53:43 +01:00
<term><varname>MODULEDIR</varname></term>
2011-02-08 22:08:41 +01:00
<listitem>
<para>
2017-10-09 03:44:17 +02:00
subdirectory of <literal><replaceable>prefix</replaceable>/share</literal>
2011-02-12 04:53:43 +01:00
into which DATA and DOCS files should be installed
(if not set, default is <literal>extension</literal> if
<varname>EXTENSION</varname> is set,
or <literal>contrib</literal> if not)
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-data">
2011-02-08 22:08:41 +01:00
<term><varname>DATA</varname></term>
<listitem>
<para>
random files to install into <literal><replaceable>prefix</replaceable>/share/$MODULEDIR</literal>
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-data-built">
2011-02-08 22:08:41 +01:00
<term><varname>DATA_built</varname></term>
<listitem>
<para>
random files to install into
<literal><replaceable>prefix</replaceable>/share/$MODULEDIR</literal>,
which need to be built first
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-data-tsearch">
2011-02-08 22:08:41 +01:00
<term><varname>DATA_TSEARCH</varname></term>
<listitem>
<para>
random files to install under
<literal><replaceable>prefix</replaceable>/share/tsearch_data</literal>
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-docs">
2011-02-08 22:08:41 +01:00
<term><varname>DOCS</varname></term>
<listitem>
<para>
random files to install under
<literal><replaceable>prefix</replaceable>/doc/$MODULEDIR</literal>
</para>
</listitem>
</varlistentry>
2018-07-31 20:58:39 +02:00
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-headers">
2018-07-31 20:58:39 +02:00
<term><varname>HEADERS</varname></term>
2018-09-05 23:01:21 +02:00
<term><varname>HEADERS_built</varname></term>
2018-07-31 20:58:39 +02:00
<listitem>
<para>
2018-09-07 14:51:30 +02:00
Files to (optionally build and) install under
<literal><replaceable>prefix</replaceable>/include/server/$MODULEDIR/$MODULE_big</literal>.
</para>
<para>
Unlike <literal>DATA_built</literal>, files in <literal>HEADERS_built</literal>
are not removed by the <literal>clean</literal> target; if you want them removed,
also add them to <literal>EXTRA_CLEAN</literal> or add your own rules to do it.
2018-07-31 20:58:39 +02:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-headers-module">
2018-07-31 20:58:39 +02:00
<term><varname>HEADERS_$MODULE</varname></term>
2018-09-05 23:01:21 +02:00
<term><varname>HEADERS_built_$MODULE</varname></term>
2018-07-31 20:58:39 +02:00
<listitem>
<para>
2018-09-05 23:01:21 +02:00
Files to install (after building if specified) under
2018-07-31 20:58:39 +02:00
<literal><replaceable>prefix</replaceable>/include/server/$MODULEDIR/$MODULE</literal>,
where <literal>$MODULE</literal> must be a module name used
2018-09-05 23:01:21 +02:00
in <literal>MODULES</literal> or <literal>MODULE_big</literal>.
</para>
2018-09-07 14:51:30 +02:00
<para>
Unlike <literal>DATA_built</literal>, files in <literal>HEADERS_built_$MODULE</literal>
are not removed by the <literal>clean</literal> target; if you want them removed,
also add them to <literal>EXTRA_CLEAN</literal> or add your own rules to do it.
</para>
2018-09-05 23:01:21 +02:00
<para>
It is legal to use both variables for the same module, or any
combination, unless you have two module names in the
<literal>MODULES</literal> list that differ only by the presence of a
prefix <literal>built_</literal>, which would cause ambiguity. In
that (hopefully unlikely) case, you should use only the
<literal>HEADERS_built_$MODULE</literal> variables.
2018-07-31 20:58:39 +02:00
</para>
</listitem>
</varlistentry>
2011-02-08 22:08:41 +01:00
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-scripts">
2011-02-08 22:08:41 +01:00
<term><varname>SCRIPTS</varname></term>
<listitem>
<para>
script files (not binaries) to install into
<literal><replaceable>prefix</replaceable>/bin</literal>
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-scripts-built">
2011-02-08 22:08:41 +01:00
<term><varname>SCRIPTS_built</varname></term>
<listitem>
<para>
script files (not binaries) to install into
<literal><replaceable>prefix</replaceable>/bin</literal>,
which need to be built first
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-regress">
2011-02-08 22:08:41 +01:00
<term><varname>REGRESS</varname></term>
<listitem>
<para>
list of regression test cases (without suffix), see below
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-regress-opts">
2011-08-24 21:16:17 +02:00
<term><varname>REGRESS_OPTS</varname></term>
<listitem>
<para>
2017-10-09 03:44:17 +02:00
additional switches to pass to <application>pg_regress</application>
2011-08-24 21:16:17 +02:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-isolation">
Add PGXS options to control TAP and isolation tests, take two
The following options are added for extensions:
- TAP_TESTS, to allow an extention to run TAP tests which are the ones
present in t/*.pl. A subset of tests can always be run with the
existing PROVE_TESTS for developers.
- ISOLATION, to define a list of isolation tests.
- ISOLATION_OPTS, to pass custom options to isolation_tester.
A couple of custom Makefile rules have been accumulated across the tree
to cover the lack of facility in PGXS for a couple of releases when
using those test suites, which are all now replaced with the new flags,
without reducing the test coverage. Note that tests of contrib/bloom/
are not enabled yet, as those are proving unstable in the buildfarm.
Author: Michael Paquier
Reviewed-by: Adam Berlin, Álvaro Herrera, Tom Lane, Nikolay Shaplov,
Arthur Zakirov
Discussion: https://postgr.es/m/20180906014849.GG2726@paquier.xyz
2018-12-03 01:27:35 +01:00
<term><varname>ISOLATION</varname></term>
<listitem>
<para>
list of isolation test cases, see below for more details
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-isolation-opts">
Add PGXS options to control TAP and isolation tests, take two
The following options are added for extensions:
- TAP_TESTS, to allow an extention to run TAP tests which are the ones
present in t/*.pl. A subset of tests can always be run with the
existing PROVE_TESTS for developers.
- ISOLATION, to define a list of isolation tests.
- ISOLATION_OPTS, to pass custom options to isolation_tester.
A couple of custom Makefile rules have been accumulated across the tree
to cover the lack of facility in PGXS for a couple of releases when
using those test suites, which are all now replaced with the new flags,
without reducing the test coverage. Note that tests of contrib/bloom/
are not enabled yet, as those are proving unstable in the buildfarm.
Author: Michael Paquier
Reviewed-by: Adam Berlin, Álvaro Herrera, Tom Lane, Nikolay Shaplov,
Arthur Zakirov
Discussion: https://postgr.es/m/20180906014849.GG2726@paquier.xyz
2018-12-03 01:27:35 +01:00
<term><varname>ISOLATION_OPTS</varname></term>
<listitem>
<para>
additional switches to pass to
<application>pg_isolation_regress</application>
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-tap-tests">
Add PGXS options to control TAP and isolation tests, take two
The following options are added for extensions:
- TAP_TESTS, to allow an extention to run TAP tests which are the ones
present in t/*.pl. A subset of tests can always be run with the
existing PROVE_TESTS for developers.
- ISOLATION, to define a list of isolation tests.
- ISOLATION_OPTS, to pass custom options to isolation_tester.
A couple of custom Makefile rules have been accumulated across the tree
to cover the lack of facility in PGXS for a couple of releases when
using those test suites, which are all now replaced with the new flags,
without reducing the test coverage. Note that tests of contrib/bloom/
are not enabled yet, as those are proving unstable in the buildfarm.
Author: Michael Paquier
Reviewed-by: Adam Berlin, Álvaro Herrera, Tom Lane, Nikolay Shaplov,
Arthur Zakirov
Discussion: https://postgr.es/m/20180906014849.GG2726@paquier.xyz
2018-12-03 01:27:35 +01:00
<term><varname>TAP_TESTS</varname></term>
<listitem>
<para>
switch defining if TAP tests need to be run, see below
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-no-install">
2021-05-27 13:58:13 +02:00
<term><varname>NO_INSTALL</varname></term>
<listitem>
<para>
don't define an <literal>install</literal> target, useful for test
modules that don't need their build products to be installed
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-no-installcheck">
2016-11-14 23:53:07 +01:00
<term><varname>NO_INSTALLCHECK</varname></term>
<listitem>
<para>
2020-09-01 00:33:37 +02:00
don't define an <literal>installcheck</literal> target, useful e.g., if tests require special configuration, or don't use <application>pg_regress</application>
2016-11-14 23:53:07 +01:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-extra-clean">
2011-02-08 22:08:41 +01:00
<term><varname>EXTRA_CLEAN</varname></term>
<listitem>
<para>
extra files to remove in <literal>make clean</literal>
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-pg-cppflags">
2011-02-08 22:08:41 +01:00
<term><varname>PG_CPPFLAGS</varname></term>
<listitem>
<para>
2019-02-03 09:48:09 +01:00
will be prepended to <varname>CPPFLAGS</varname>
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-pg-cflags">
2019-02-03 09:48:09 +01:00
<term><varname>PG_CFLAGS</varname></term>
<listitem>
<para>
will be appended to <varname>CFLAGS</varname>
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-pg-cxxflags">
2019-02-03 09:48:09 +01:00
<term><varname>PG_CXXFLAGS</varname></term>
<listitem>
<para>
will be appended to <varname>CXXFLAGS</varname>
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-pg-ldflags">
2019-02-03 09:48:09 +01:00
<term><varname>PG_LDFLAGS</varname></term>
<listitem>
<para>
will be prepended to <varname>LDFLAGS</varname>
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-pg-libs">
2011-02-08 22:08:41 +01:00
<term><varname>PG_LIBS</varname></term>
<listitem>
<para>
will be added to <varname>PROGRAM</varname> link line
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-shlib-link">
2011-02-08 22:08:41 +01:00
<term><varname>SHLIB_LINK</varname></term>
<listitem>
<para>
will be added to <varname>MODULE_big</varname> link line
</para>
</listitem>
</varlistentry>
2023-01-09 21:08:24 +01:00
<varlistentry id="extend-pgxs-pg-config">
2011-02-08 22:08:41 +01:00
<term><varname>PG_CONFIG</varname></term>
<listitem>
<para>
2017-10-09 03:44:17 +02:00
path to <application>pg_config</application> program for the
2011-02-08 22:08:41 +01:00
<productname>PostgreSQL</productname> installation to build against
2017-10-09 03:44:17 +02:00
(typically just <literal>pg_config</literal> to use the first one in your
<varname>PATH</varname>)
2011-02-08 22:08:41 +01:00
</para>
</listitem>
</varlistentry>
</variablelist>
</para>
<para>
Put this makefile as <literal>Makefile</literal> in the directory
which holds your extension. Then you can do
<literal>make</literal> to compile, and then <literal>make
install</literal> to install your module. By default, the extension is
compiled and installed for the
<productname>PostgreSQL</productname> installation that
corresponds to the first <command>pg_config</command> program
2017-10-09 03:44:17 +02:00
found in your <varname>PATH</varname>. You can use a different installation by
2011-02-08 22:08:41 +01:00
setting <varname>PG_CONFIG</varname> to point to its
<command>pg_config</command> program, either within the makefile
or on the <literal>make</literal> command line.
</para>
2013-10-07 04:57:16 +02:00
<para>
You can also run <literal>make</literal> in a directory outside the source
tree of your extension, if you want to keep the build directory separate.
This procedure is also called a
<indexterm><primary>VPATH</primary></indexterm><firstterm>VPATH</firstterm>
build. Here's how:
2014-07-08 17:39:07 +02:00
<programlisting>
mkdir build_dir
cd build_dir
make -f /path/to/extension/source/tree/Makefile
make -f /path/to/extension/source/tree/Makefile install
</programlisting>
2013-10-07 04:57:16 +02:00
</para>
<para>
Alternatively, you can set up a directory for a VPATH build in a similar
2013-12-13 18:58:48 +01:00
way to how it is done for the core code. One way to do this is using the
2017-10-09 03:44:17 +02:00
core script <filename>config/prep_buildtree</filename>. Once this has been done
2013-10-07 04:57:16 +02:00
you can build by setting the <literal>make</literal> variable
2014-11-20 03:51:30 +01:00
<varname>VPATH</varname> like this:
2014-07-08 17:39:07 +02:00
<programlisting>
2014-11-20 03:51:30 +01:00
make VPATH=/path/to/extension/source/tree
make VPATH=/path/to/extension/source/tree install
2014-07-08 17:39:07 +02:00
</programlisting>
2013-10-07 04:57:16 +02:00
This procedure can work with a greater variety of directory layouts.
</para>
2011-02-08 22:08:41 +01:00
<para>
2017-10-09 03:44:17 +02:00
The scripts listed in the <varname>REGRESS</varname> variable are used for
2011-02-08 22:08:41 +01:00
regression testing of your module, which can be invoked by <literal>make
2017-10-09 03:44:17 +02:00
installcheck</literal> after doing <literal>make install</literal>. For this to
2011-02-08 22:08:41 +01:00
work you must have a running <productname>PostgreSQL</productname> server.
2017-10-09 03:44:17 +02:00
The script files listed in <varname>REGRESS</varname> must appear in a
2011-02-08 22:08:41 +01:00
subdirectory named <literal>sql/</literal> in your extension's directory.
These files must have extension <literal>.sql</literal>, which must not be
included in the <varname>REGRESS</varname> list in the makefile. For each
test there should also be a file containing the expected output in a
subdirectory named <literal>expected/</literal>, with the same stem and
extension <literal>.out</literal>. <literal>make installcheck</literal>
2017-10-09 03:44:17 +02:00
executes each test script with <application>psql</application>, and compares the
2011-02-08 22:08:41 +01:00
resulting output to the matching expected file. Any differences will be
written to the file <literal>regression.diffs</literal> in <command>diff
-c</command> format. Note that trying to run a test that is missing its
expected file will be reported as <quote>trouble</quote>, so make sure you
have all expected files.
</para>
Add PGXS options to control TAP and isolation tests, take two
The following options are added for extensions:
- TAP_TESTS, to allow an extention to run TAP tests which are the ones
present in t/*.pl. A subset of tests can always be run with the
existing PROVE_TESTS for developers.
- ISOLATION, to define a list of isolation tests.
- ISOLATION_OPTS, to pass custom options to isolation_tester.
A couple of custom Makefile rules have been accumulated across the tree
to cover the lack of facility in PGXS for a couple of releases when
using those test suites, which are all now replaced with the new flags,
without reducing the test coverage. Note that tests of contrib/bloom/
are not enabled yet, as those are proving unstable in the buildfarm.
Author: Michael Paquier
Reviewed-by: Adam Berlin, Álvaro Herrera, Tom Lane, Nikolay Shaplov,
Arthur Zakirov
Discussion: https://postgr.es/m/20180906014849.GG2726@paquier.xyz
2018-12-03 01:27:35 +01:00
<para>
The scripts listed in the <varname>ISOLATION</varname> variable are used
for tests stressing behavior of concurrent session with your module, which
can be invoked by <literal>make installcheck</literal> after doing
<literal>make install</literal>. For this to work you must have a
running <productname>PostgreSQL</productname> server. The script files
listed in <varname>ISOLATION</varname> must appear in a subdirectory
named <literal>specs/</literal> in your extension's directory. These files
must have extension <literal>.spec</literal>, which must not be included
in the <varname>ISOLATION</varname> list in the makefile. For each test
there should also be a file containing the expected output in a
subdirectory named <literal>expected/</literal>, with the same stem and
extension <literal>.out</literal>. <literal>make installcheck</literal>
executes each test script, and compares the resulting output to the
matching expected file. Any differences will be written to the file
<literal>output_iso/regression.diffs</literal> in
<command>diff -c</command> format. Note that trying to run a test that is
missing its expected file will be reported as <quote>trouble</quote>, so
make sure you have all expected files.
</para>
<para>
<literal>TAP_TESTS</literal> enables the use of TAP tests. Data from each
run is present in a subdirectory named <literal>tmp_check/</literal>.
See also <xref linkend="regress-tap"/> for more details.
</para>
2011-02-08 22:08:41 +01:00
<tip>
<para>
The easiest way to create the expected files is to create empty files,
then do a test run (which will of course report differences). Inspect
the actual result files found in the <literal>results/</literal>
Add PGXS options to control TAP and isolation tests, take two
The following options are added for extensions:
- TAP_TESTS, to allow an extention to run TAP tests which are the ones
present in t/*.pl. A subset of tests can always be run with the
existing PROVE_TESTS for developers.
- ISOLATION, to define a list of isolation tests.
- ISOLATION_OPTS, to pass custom options to isolation_tester.
A couple of custom Makefile rules have been accumulated across the tree
to cover the lack of facility in PGXS for a couple of releases when
using those test suites, which are all now replaced with the new flags,
without reducing the test coverage. Note that tests of contrib/bloom/
are not enabled yet, as those are proving unstable in the buildfarm.
Author: Michael Paquier
Reviewed-by: Adam Berlin, Álvaro Herrera, Tom Lane, Nikolay Shaplov,
Arthur Zakirov
Discussion: https://postgr.es/m/20180906014849.GG2726@paquier.xyz
2018-12-03 01:27:35 +01:00
directory (for tests in <literal>REGRESS</literal>), or
<literal>output_iso/results/</literal> directory (for tests in
<literal>ISOLATION</literal>), then copy them to
<literal>expected/</literal> if they match what you expect from the test.
2011-02-08 22:08:41 +01:00
</para>
</tip>
2010-06-01 04:31:36 +02:00
</sect1>
1999-07-22 17:11:05 +02:00
</chapter>