2017-01-19 18:00:00 +01:00
|
|
|
<!--
|
|
|
|
|
doc/src/sgml/ref/create_subscription.sgml
|
|
|
|
|
PostgreSQL documentation
|
|
|
|
|
-->
|
|
|
|
|
|
2017-10-20 03:16:39 +02:00
|
|
|
<refentry id="sql-createsubscription">
|
2017-01-19 18:00:00 +01:00
|
|
|
<indexterm zone="sql-createsubscription">
|
|
|
|
|
<primary>CREATE SUBSCRIPTION</primary>
|
|
|
|
|
</indexterm>
|
|
|
|
|
|
|
|
|
|
<refmeta>
|
|
|
|
|
<refentrytitle>CREATE SUBSCRIPTION</refentrytitle>
|
|
|
|
|
<manvolnum>7</manvolnum>
|
|
|
|
|
<refmiscinfo>SQL - Language Statements</refmiscinfo>
|
|
|
|
|
</refmeta>
|
|
|
|
|
|
|
|
|
|
<refnamediv>
|
|
|
|
|
<refname>CREATE SUBSCRIPTION</refname>
|
|
|
|
|
<refpurpose>define a new subscription</refpurpose>
|
|
|
|
|
</refnamediv>
|
|
|
|
|
|
|
|
|
|
<refsynopsisdiv>
|
|
|
|
|
<synopsis>
|
2017-10-09 04:00:57 +02:00
|
|
|
CREATE SUBSCRIPTION <replaceable class="parameter">subscription_name</replaceable>
|
|
|
|
|
CONNECTION '<replaceable class="parameter">conninfo</replaceable>'
|
|
|
|
|
PUBLICATION <replaceable class="parameter">publication_name</replaceable> [, ...]
|
2017-05-12 14:57:01 +02:00
|
|
|
[ WITH ( <replaceable class="parameter">subscription_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) ]
|
2017-01-19 18:00:00 +01:00
|
|
|
</synopsis>
|
|
|
|
|
</refsynopsisdiv>
|
|
|
|
|
|
|
|
|
|
<refsect1>
|
|
|
|
|
<title>Description</title>
|
|
|
|
|
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
<command>CREATE SUBSCRIPTION</command> adds a new logical-replication
|
Add new predefined role pg_create_subscription.
This role can be granted to non-superusers to allow them to issue
CREATE SUBSCRIPTION. The non-superuser must additionally have CREATE
permissions on the database in which the subscription is to be
created.
Most forms of ALTER SUBSCRIPTION, including ALTER SUBSCRIPTION .. SKIP,
now require only that the role performing the operation own the
subscription, or inherit the privileges of the owner. However, to
use ALTER SUBSCRIPTION ... RENAME or ALTER SUBSCRIPTION ... OWNER TO,
you also need CREATE permission on the database. This is similar to
what we do for schemas. To change the owner of a schema, you must also
have permission to SET ROLE to the new owner, similar to what we do
for other object types.
Non-superusers are required to specify a password for authentication
and the remote side must use the password, similar to what is required
for postgres_fdw and dblink. A superuser who wants a non-superuser to
own a subscription that does not rely on password authentication may
set the new password_required=false property on that subscription. A
non-superuser may not set password_required=false and may not modify a
subscription that already has password_required=false.
This new password_required subscription property works much like the
eponymous postgres_fdw property. In both cases, the actual semantics
are that a password is not required if either (1) the property is set
to false or (2) the relevant user is the superuser.
Patch by me, reviewed by Andres Freund, Jeff Davis, Mark Dilger,
and Stephen Frost (but some of those people did not fully endorse
all of the decisions that the patch makes).
Discussion: http://postgr.es/m/CA+TgmoaDH=0Xj7OBiQnsHTKcF2c4L+=gzPBUKSJLh8zed2_+Dg@mail.gmail.com
2023-03-30 17:37:19 +02:00
|
|
|
subscription. The user that creates a subscription becomes the owner
|
|
|
|
|
of the subscription. The subscription name must be distinct from the name of
|
2021-09-13 20:27:02 +02:00
|
|
|
any existing subscription in the current database.
|
2017-01-19 18:00:00 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
A subscription represents a replication connection to the publisher.
|
|
|
|
|
Hence, in addition to adding definitions in the local catalogs, this
|
|
|
|
|
command normally creates a replication slot on the publisher.
|
2017-01-19 18:00:00 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
A logical replication worker will be started to replicate data for the new
|
2021-09-13 20:27:02 +02:00
|
|
|
subscription at the commit of the transaction where this command is run,
|
|
|
|
|
unless the subscription is initially disabled.
|
2017-01-19 18:00:00 +01:00
|
|
|
</para>
|
|
|
|
|
|
Add new predefined role pg_create_subscription.
This role can be granted to non-superusers to allow them to issue
CREATE SUBSCRIPTION. The non-superuser must additionally have CREATE
permissions on the database in which the subscription is to be
created.
Most forms of ALTER SUBSCRIPTION, including ALTER SUBSCRIPTION .. SKIP,
now require only that the role performing the operation own the
subscription, or inherit the privileges of the owner. However, to
use ALTER SUBSCRIPTION ... RENAME or ALTER SUBSCRIPTION ... OWNER TO,
you also need CREATE permission on the database. This is similar to
what we do for schemas. To change the owner of a schema, you must also
have permission to SET ROLE to the new owner, similar to what we do
for other object types.
Non-superusers are required to specify a password for authentication
and the remote side must use the password, similar to what is required
for postgres_fdw and dblink. A superuser who wants a non-superuser to
own a subscription that does not rely on password authentication may
set the new password_required=false property on that subscription. A
non-superuser may not set password_required=false and may not modify a
subscription that already has password_required=false.
This new password_required subscription property works much like the
eponymous postgres_fdw property. In both cases, the actual semantics
are that a password is not required if either (1) the property is set
to false or (2) the relevant user is the superuser.
Patch by me, reviewed by Andres Freund, Jeff Davis, Mark Dilger,
and Stephen Frost (but some of those people did not fully endorse
all of the decisions that the patch makes).
Discussion: http://postgr.es/m/CA+TgmoaDH=0Xj7OBiQnsHTKcF2c4L+=gzPBUKSJLh8zed2_+Dg@mail.gmail.com
2023-03-30 17:37:19 +02:00
|
|
|
<para>
|
2023-10-09 05:48:47 +02:00
|
|
|
To be able to create a subscription, you must have the privileges of
|
Add new predefined role pg_create_subscription.
This role can be granted to non-superusers to allow them to issue
CREATE SUBSCRIPTION. The non-superuser must additionally have CREATE
permissions on the database in which the subscription is to be
created.
Most forms of ALTER SUBSCRIPTION, including ALTER SUBSCRIPTION .. SKIP,
now require only that the role performing the operation own the
subscription, or inherit the privileges of the owner. However, to
use ALTER SUBSCRIPTION ... RENAME or ALTER SUBSCRIPTION ... OWNER TO,
you also need CREATE permission on the database. This is similar to
what we do for schemas. To change the owner of a schema, you must also
have permission to SET ROLE to the new owner, similar to what we do
for other object types.
Non-superusers are required to specify a password for authentication
and the remote side must use the password, similar to what is required
for postgres_fdw and dblink. A superuser who wants a non-superuser to
own a subscription that does not rely on password authentication may
set the new password_required=false property on that subscription. A
non-superuser may not set password_required=false and may not modify a
subscription that already has password_required=false.
This new password_required subscription property works much like the
eponymous postgres_fdw property. In both cases, the actual semantics
are that a password is not required if either (1) the property is set
to false or (2) the relevant user is the superuser.
Patch by me, reviewed by Andres Freund, Jeff Davis, Mark Dilger,
and Stephen Frost (but some of those people did not fully endorse
all of the decisions that the patch makes).
Discussion: http://postgr.es/m/CA+TgmoaDH=0Xj7OBiQnsHTKcF2c4L+=gzPBUKSJLh8zed2_+Dg@mail.gmail.com
2023-03-30 17:37:19 +02:00
|
|
|
the <literal>pg_create_subscription</literal> role, as well as
|
|
|
|
|
<literal>CREATE</literal> privileges on the current database.
|
|
|
|
|
</para>
|
|
|
|
|
|
2017-01-19 18:00:00 +01:00
|
|
|
<para>
|
2018-11-03 17:21:54 +01:00
|
|
|
Additional information about subscriptions and logical replication as a
|
|
|
|
|
whole is available at <xref linkend="logical-replication-subscription"/> and
|
2017-11-23 15:39:47 +01:00
|
|
|
<xref linkend="logical-replication"/>.
|
2017-01-19 18:00:00 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
</refsect1>
|
|
|
|
|
|
|
|
|
|
<refsect1>
|
|
|
|
|
<title>Parameters</title>
|
|
|
|
|
|
|
|
|
|
<variablelist>
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-name">
|
2017-01-19 18:00:00 +01:00
|
|
|
<term><replaceable class="parameter">subscription_name</replaceable></term>
|
|
|
|
|
<listitem>
|
|
|
|
|
<para>
|
|
|
|
|
The name of the new subscription.
|
|
|
|
|
</para>
|
|
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
|
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-connection">
|
2017-01-19 18:00:00 +01:00
|
|
|
<term><literal>CONNECTION '<replaceable class="parameter">conninfo</replaceable>'</literal></term>
|
|
|
|
|
<listitem>
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
The <application>libpq</application> connection string defining how
|
|
|
|
|
to connect to the publisher database. For details see
|
|
|
|
|
<xref linkend="libpq-connstring"/>.
|
2017-01-19 18:00:00 +01:00
|
|
|
</para>
|
|
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
|
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-publication">
|
2021-09-13 20:27:02 +02:00
|
|
|
<term><literal>PUBLICATION <replaceable class="parameter">publication_name</replaceable> [, ...]</literal></term>
|
2017-01-19 18:00:00 +01:00
|
|
|
<listitem>
|
|
|
|
|
<para>
|
2017-05-12 14:57:01 +02:00
|
|
|
Names of the publications on the publisher to subscribe to.
|
2017-03-23 13:36:36 +01:00
|
|
|
</para>
|
|
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
|
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with">
|
2017-05-12 14:57:01 +02:00
|
|
|
<term><literal>WITH ( <replaceable class="parameter">subscription_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] )</literal></term>
|
2017-04-14 19:58:46 +02:00
|
|
|
<listitem>
|
|
|
|
|
<para>
|
2021-09-13 04:59:10 +02:00
|
|
|
This clause specifies optional parameters for a subscription.
|
|
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
The following parameters control what happens during subscription creation:
|
2017-05-12 14:57:01 +02:00
|
|
|
|
|
|
|
|
<variablelist>
|
2021-09-13 04:59:10 +02:00
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-connect">
|
2021-09-13 04:59:10 +02:00
|
|
|
<term><literal>connect</literal> (<type>boolean</type>)</term>
|
2017-05-12 14:57:01 +02:00
|
|
|
<listitem>
|
|
|
|
|
<para>
|
2021-09-13 04:59:10 +02:00
|
|
|
Specifies whether the <command>CREATE SUBSCRIPTION</command>
|
2021-09-13 20:27:02 +02:00
|
|
|
command should connect to the publisher at all. The default
|
|
|
|
|
is <literal>true</literal>. Setting this to
|
|
|
|
|
<literal>false</literal> will force the values of
|
|
|
|
|
<literal>create_slot</literal>, <literal>enabled</literal> and
|
2021-09-13 04:59:10 +02:00
|
|
|
<literal>copy_data</literal> to <literal>false</literal>.
|
2021-09-13 20:27:02 +02:00
|
|
|
(You cannot combine setting <literal>connect</literal>
|
|
|
|
|
to <literal>false</literal> with
|
|
|
|
|
setting <literal>create_slot</literal>, <literal>enabled</literal>,
|
|
|
|
|
or <literal>copy_data</literal> to <literal>true</literal>.)
|
2021-09-13 04:59:10 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
Since no connection is made when this option is
|
2022-11-02 07:10:37 +01:00
|
|
|
<literal>false</literal>, no tables are subscribed. To initiate
|
|
|
|
|
replication, you must manually create the replication slot, enable
|
|
|
|
|
the subscription, and refresh the subscription. See
|
|
|
|
|
<xref linkend="logical-replication-subscription-examples-deferred-slot"/>
|
|
|
|
|
for examples.
|
2017-05-12 14:57:01 +02:00
|
|
|
</para>
|
|
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
|
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-create-slot">
|
2017-05-12 14:57:01 +02:00
|
|
|
<term><literal>create_slot</literal> (<type>boolean</type>)</term>
|
|
|
|
|
<listitem>
|
|
|
|
|
<para>
|
|
|
|
|
Specifies whether the command should create the replication slot on
|
|
|
|
|
the publisher. The default is <literal>true</literal>.
|
2022-11-02 07:10:37 +01:00
|
|
|
</para>
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
If set to <literal>false</literal>, you are responsible for
|
2022-11-02 07:10:37 +01:00
|
|
|
creating the publisher's slot in some other way. See
|
|
|
|
|
<xref linkend="logical-replication-subscription-examples-deferred-slot"/>
|
|
|
|
|
for examples.
|
2017-05-12 14:57:01 +02:00
|
|
|
</para>
|
|
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
|
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-enabled">
|
2017-05-12 14:57:01 +02:00
|
|
|
<term><literal>enabled</literal> (<type>boolean</type>)</term>
|
|
|
|
|
<listitem>
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
Specifies whether the subscription should be actively replicating
|
|
|
|
|
or whether it should just be set up but not started yet. The default
|
2017-05-12 14:57:01 +02:00
|
|
|
is <literal>true</literal>.
|
|
|
|
|
</para>
|
|
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
|
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-slot-name">
|
2017-05-12 14:57:01 +02:00
|
|
|
<term><literal>slot_name</literal> (<type>string</type>)</term>
|
|
|
|
|
<listitem>
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
Name of the publisher's replication slot to use. The default is
|
|
|
|
|
to use the name of the subscription for the slot name.
|
2017-05-12 14:57:01 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
Setting <literal>slot_name</literal> to <literal>NONE</literal>
|
2022-11-02 07:10:37 +01:00
|
|
|
means there will be no replication slot associated with the
|
|
|
|
|
subscription. Such subscriptions must also have both
|
|
|
|
|
<literal>enabled</literal> and <literal>create_slot</literal> set to
|
|
|
|
|
<literal>false</literal>. Use this when you will be creating the
|
|
|
|
|
replication slot later manually. See
|
|
|
|
|
<xref linkend="logical-replication-subscription-examples-deferred-slot"/>
|
|
|
|
|
for examples.
|
2017-05-12 14:57:01 +02:00
|
|
|
</para>
|
|
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
2021-09-13 04:59:10 +02:00
|
|
|
</variablelist>
|
|
|
|
|
</para>
|
2017-05-12 14:57:01 +02:00
|
|
|
|
2021-09-13 04:59:10 +02:00
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
The following parameters control the subscription's replication
|
|
|
|
|
behavior after it has been created:
|
2017-05-12 14:57:01 +02:00
|
|
|
|
2021-09-13 04:59:10 +02:00
|
|
|
<variablelist>
|
2017-05-12 14:57:01 +02:00
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-binary">
|
2020-07-18 18:44:51 +02:00
|
|
|
<term><literal>binary</literal> (<type>boolean</type>)</term>
|
|
|
|
|
<listitem>
|
|
|
|
|
<para>
|
2023-03-23 04:15:51 +01:00
|
|
|
Specifies whether the subscription will request the publisher to send
|
|
|
|
|
the data in binary format (as opposed to text). The default is
|
|
|
|
|
<literal>false</literal>. Any initial table synchronization copy
|
|
|
|
|
(see <literal>copy_data</literal>) also uses the same format. Binary
|
|
|
|
|
format can be faster than the text format, but it is less portable
|
|
|
|
|
across machine architectures and <productname>PostgreSQL</productname>
|
|
|
|
|
versions. Binary format is very data type specific; for example, it
|
|
|
|
|
will not allow copying from a <type>smallint</type> column to an
|
|
|
|
|
<type>integer</type> column, even though that would work fine in text
|
|
|
|
|
format. Even when this option is enabled, only data types having binary
|
|
|
|
|
send and receive functions will be transferred in binary. Note that
|
|
|
|
|
the initial synchronization requires all data types to have binary
|
|
|
|
|
send and receive functions, otherwise the synchronization will fail
|
|
|
|
|
(see <xref linkend="sql-createtype"/> for more about send/receive
|
|
|
|
|
functions).
|
2020-07-18 18:44:51 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
When doing cross-version replication, it could be that the
|
2020-07-18 18:44:51 +02:00
|
|
|
publisher has a binary send function for some data type, but the
|
2021-09-13 20:27:02 +02:00
|
|
|
subscriber lacks a binary receive function for that type. In
|
2020-07-18 18:44:51 +02:00
|
|
|
such a case, data transfer will fail, and
|
|
|
|
|
the <literal>binary</literal> option cannot be used.
|
|
|
|
|
</para>
|
2023-03-23 04:15:51 +01:00
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
If the publisher is a <productname>PostgreSQL</productname> version
|
|
|
|
|
before 16, then any initial table synchronization will use text format
|
|
|
|
|
even if <literal>binary = true</literal>.
|
|
|
|
|
</para>
|
2020-07-18 18:44:51 +02:00
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
|
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-copy-data">
|
2021-09-13 04:59:10 +02:00
|
|
|
<term><literal>copy_data</literal> (<type>boolean</type>)</term>
|
2017-05-12 14:57:01 +02:00
|
|
|
<listitem>
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
Specifies whether to copy pre-existing data in the publications
|
|
|
|
|
that are being subscribed to when the replication starts.
|
2021-09-13 04:59:10 +02:00
|
|
|
The default is <literal>true</literal>.
|
2017-05-12 14:57:01 +02:00
|
|
|
</para>
|
Allow specifying row filters for logical replication of tables.
This feature adds row filtering for publication tables. When a publication
is defined or modified, an optional WHERE clause can be specified. Rows
that don't satisfy this WHERE clause will be filtered out. This allows a
set of tables to be partially replicated. The row filter is per table. A
new row filter can be added simply by specifying a WHERE clause after the
table name. The WHERE clause must be enclosed by parentheses.
The row filter WHERE clause for a table added to a publication that
publishes UPDATE and/or DELETE operations must contain only columns that
are covered by REPLICA IDENTITY. The row filter WHERE clause for a table
added to a publication that publishes INSERT can use any column. If the
row filter evaluates to NULL, it is regarded as "false". The WHERE clause
only allows simple expressions that don't have user-defined functions,
user-defined operators, user-defined types, user-defined collations,
non-immutable built-in functions, or references to system columns. These
restrictions could be addressed in the future.
If you choose to do the initial table synchronization, only data that
satisfies the row filters is copied to the subscriber. If the subscription
has several publications in which a table has been published with
different WHERE clauses, rows that satisfy ANY of the expressions will be
copied. If a subscriber is a pre-15 version, the initial table
synchronization won't use row filters even if they are defined in the
publisher.
The row filters are applied before publishing the changes. If the
subscription has several publications in which the same table has been
published with different filters (for the same publish operation), those
expressions get OR'ed together so that rows satisfying any of the
expressions will be replicated.
This means all the other filters become redundant if (a) one of the
publications have no filter at all, (b) one of the publications was
created using FOR ALL TABLES, (c) one of the publications was created
using FOR ALL TABLES IN SCHEMA and the table belongs to that same schema.
If your publication contains a partitioned table, the publication
parameter publish_via_partition_root determines if it uses the partition's
row filter (if the parameter is false, the default) or the root
partitioned table's row filter.
Psql commands \dRp+ and \d <table-name> will display any row filters.
Author: Hou Zhijie, Euler Taveira, Peter Smith, Ajin Cherian
Reviewed-by: Greg Nancarrow, Haiying Tang, Amit Kapila, Tomas Vondra, Dilip Kumar, Vignesh C, Alvaro Herrera, Andres Freund, Wei Wang
Discussion: https://www.postgresql.org/message-id/flat/CAHE3wggb715X%2BmK_DitLXF25B%3DjE6xyNCH4YOwM860JR7HarGQ%40mail.gmail.com
2022-02-22 03:24:12 +01:00
|
|
|
<para>
|
|
|
|
|
If the publications contain <literal>WHERE</literal> clauses, it
|
|
|
|
|
will affect what data is copied. Refer to the
|
|
|
|
|
<xref linkend="sql-createsubscription-notes" /> for details.
|
|
|
|
|
</para>
|
2022-09-08 03:24:13 +02:00
|
|
|
<para>
|
|
|
|
|
See <xref linkend="sql-createsubscription-notes"/> for details of how
|
|
|
|
|
<literal>copy_data = true</literal> can interact with the
|
|
|
|
|
<literal>origin</literal> parameter.
|
|
|
|
|
</para>
|
2017-05-12 14:57:01 +02:00
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
2021-09-13 04:59:10 +02:00
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-streaming">
|
Perform apply of large transactions by parallel workers.
Currently, for large transactions, the publisher sends the data in
multiple streams (changes divided into chunks depending upon
logical_decoding_work_mem), and then on the subscriber-side, the apply
worker writes the changes into temporary files and once it receives the
commit, it reads from those files and applies the entire transaction. To
improve the performance of such transactions, we can instead allow them to
be applied via parallel workers.
In this approach, we assign a new parallel apply worker (if available) as
soon as the xact's first stream is received and the leader apply worker
will send changes to this new worker via shared memory. The parallel apply
worker will directly apply the change instead of writing it to temporary
files. However, if the leader apply worker times out while attempting to
send a message to the parallel apply worker, it will switch to
"partial serialize" mode - in this mode, the leader serializes all
remaining changes to a file and notifies the parallel apply workers to
read and apply them at the end of the transaction. We use a non-blocking
way to send the messages from the leader apply worker to the parallel
apply to avoid deadlocks. We keep this parallel apply assigned till the
transaction commit is received and also wait for the worker to finish at
commit. This preserves commit ordering and avoid writing to and reading
from files in most cases. We still need to spill if there is no worker
available.
This patch also extends the SUBSCRIPTION 'streaming' parameter so that the
user can control whether to apply the streaming transaction in a parallel
apply worker or spill the change to disk. The user can set the streaming
parameter to 'on/off', or 'parallel'. The parameter value 'parallel' means
the streaming will be applied via a parallel apply worker, if available.
The parameter value 'on' means the streaming transaction will be spilled
to disk. The default value is 'off' (same as current behaviour).
In addition, the patch extends the logical replication STREAM_ABORT
message so that abort_lsn and abort_time can also be sent which can be
used to update the replication origin in parallel apply worker when the
streaming transaction is aborted. Because this message extension is needed
to support parallel streaming, parallel streaming is not supported for
publications on servers < PG16.
Author: Hou Zhijie, Wang wei, Amit Kapila with design inputs from Sawada Masahiko
Reviewed-by: Sawada Masahiko, Peter Smith, Dilip Kumar, Shi yu, Kuroda Hayato, Shveta Mallik
Discussion: https://postgr.es/m/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com
2023-01-09 02:30:39 +01:00
|
|
|
<term><literal>streaming</literal> (<type>enum</type>)</term>
|
Add support for streaming to built-in logical replication.
To add support for streaming of in-progress transactions into the
built-in logical replication, we need to do three things:
* Extend the logical replication protocol, so identify in-progress
transactions, and allow adding additional bits of information (e.g.
XID of subtransactions).
* Modify the output plugin (pgoutput) to implement the new stream
API callbacks, by leveraging the extended replication protocol.
* Modify the replication apply worker, to properly handle streamed
in-progress transaction by spilling the data to disk and then
replaying them on commit.
We however must explicitly disable streaming replication during
replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover we don't have a replication connection open so we
don't have where to send the data anyway.
Author: Tomas Vondra, Dilip Kumar and Amit Kapila
Reviewed-by: Amit Kapila, Kuntal Ghosh and Ajin Cherian
Tested-by: Neha Sharma, Mahendra Singh Thalor and Ajin Cherian
Discussion: https://postgr.es/m/688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com
2020-09-03 04:24:07 +02:00
|
|
|
<listitem>
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
Specifies whether to enable streaming of in-progress transactions
|
Perform apply of large transactions by parallel workers.
Currently, for large transactions, the publisher sends the data in
multiple streams (changes divided into chunks depending upon
logical_decoding_work_mem), and then on the subscriber-side, the apply
worker writes the changes into temporary files and once it receives the
commit, it reads from those files and applies the entire transaction. To
improve the performance of such transactions, we can instead allow them to
be applied via parallel workers.
In this approach, we assign a new parallel apply worker (if available) as
soon as the xact's first stream is received and the leader apply worker
will send changes to this new worker via shared memory. The parallel apply
worker will directly apply the change instead of writing it to temporary
files. However, if the leader apply worker times out while attempting to
send a message to the parallel apply worker, it will switch to
"partial serialize" mode - in this mode, the leader serializes all
remaining changes to a file and notifies the parallel apply workers to
read and apply them at the end of the transaction. We use a non-blocking
way to send the messages from the leader apply worker to the parallel
apply to avoid deadlocks. We keep this parallel apply assigned till the
transaction commit is received and also wait for the worker to finish at
commit. This preserves commit ordering and avoid writing to and reading
from files in most cases. We still need to spill if there is no worker
available.
This patch also extends the SUBSCRIPTION 'streaming' parameter so that the
user can control whether to apply the streaming transaction in a parallel
apply worker or spill the change to disk. The user can set the streaming
parameter to 'on/off', or 'parallel'. The parameter value 'parallel' means
the streaming will be applied via a parallel apply worker, if available.
The parameter value 'on' means the streaming transaction will be spilled
to disk. The default value is 'off' (same as current behaviour).
In addition, the patch extends the logical replication STREAM_ABORT
message so that abort_lsn and abort_time can also be sent which can be
used to update the replication origin in parallel apply worker when the
streaming transaction is aborted. Because this message extension is needed
to support parallel streaming, parallel streaming is not supported for
publications on servers < PG16.
Author: Hou Zhijie, Wang wei, Amit Kapila with design inputs from Sawada Masahiko
Reviewed-by: Sawada Masahiko, Peter Smith, Dilip Kumar, Shi yu, Kuroda Hayato, Shveta Mallik
Discussion: https://postgr.es/m/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com
2023-01-09 02:30:39 +01:00
|
|
|
for this subscription. The default value is <literal>off</literal>,
|
|
|
|
|
meaning all transactions are fully decoded on the publisher and only
|
|
|
|
|
then sent to the subscriber as a whole.
|
|
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
If set to <literal>on</literal>, the incoming changes are written to
|
|
|
|
|
temporary files and then applied only after the transaction is
|
|
|
|
|
committed on the publisher and received by the subscriber.
|
|
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
If set to <literal>parallel</literal>, incoming changes are directly
|
|
|
|
|
applied via one of the parallel apply workers, if available. If no
|
|
|
|
|
parallel apply worker is free to handle streaming transactions then
|
|
|
|
|
the changes are written to temporary files and applied after the
|
|
|
|
|
transaction is committed. Note that if an error happens in a
|
|
|
|
|
parallel apply worker, the finish LSN of the remote transaction
|
|
|
|
|
might not be reported in the server log.
|
Add support for streaming to built-in logical replication.
To add support for streaming of in-progress transactions into the
built-in logical replication, we need to do three things:
* Extend the logical replication protocol, so identify in-progress
transactions, and allow adding additional bits of information (e.g.
XID of subtransactions).
* Modify the output plugin (pgoutput) to implement the new stream
API callbacks, by leveraging the extended replication protocol.
* Modify the replication apply worker, to properly handle streamed
in-progress transaction by spilling the data to disk and then
replaying them on commit.
We however must explicitly disable streaming replication during
replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover we don't have a replication connection open so we
don't have where to send the data anyway.
Author: Tomas Vondra, Dilip Kumar and Amit Kapila
Reviewed-by: Amit Kapila, Kuntal Ghosh and Ajin Cherian
Tested-by: Neha Sharma, Mahendra Singh Thalor and Ajin Cherian
Discussion: https://postgr.es/m/688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com
2020-09-03 04:24:07 +02:00
|
|
|
</para>
|
2021-09-13 04:59:10 +02:00
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
|
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-synchronous-commit">
|
2021-09-13 04:59:10 +02:00
|
|
|
<term><literal>synchronous_commit</literal> (<type>enum</type>)</term>
|
|
|
|
|
<listitem>
|
|
|
|
|
<para>
|
|
|
|
|
The value of this parameter overrides the
|
|
|
|
|
<xref linkend="guc-synchronous-commit"/> setting within this
|
|
|
|
|
subscription's apply worker processes. The default value
|
|
|
|
|
is <literal>off</literal>.
|
|
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
It is safe to use <literal>off</literal> for logical replication:
|
|
|
|
|
If the subscriber loses transactions because of missing
|
|
|
|
|
synchronization, the data will be sent again from the publisher.
|
|
|
|
|
</para>
|
Add support for prepared transactions to built-in logical replication.
To add support for streaming transactions at prepare time into the
built-in logical replication, we need to do the following things:
* Modify the output plugin (pgoutput) to implement the new two-phase API
callbacks, by leveraging the extended replication protocol.
* Modify the replication apply worker, to properly handle two-phase
transactions by replaying them on prepare.
* Add a new SUBSCRIPTION option "two_phase" to allow users to enable
two-phase transactions. We enable the two_phase once the initial data sync
is over.
We however must explicitly disable replication of two-phase transactions
during replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover, we don't have a replication connection open so we don't know
where to send the data anyway.
The streaming option is not allowed with this new two_phase option. This
can be done as a separate patch.
We don't allow to toggle two_phase option of a subscription because it can
lead to an inconsistent replica. For the same reason, we don't allow to
refresh the publication once the two_phase is enabled for a subscription
unless copy_data option is false.
Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
2021-07-14 04:03:50 +02:00
|
|
|
|
2021-09-13 04:59:10 +02:00
|
|
|
<para>
|
|
|
|
|
A different setting might be appropriate when doing synchronous
|
|
|
|
|
logical replication. The logical replication workers report the
|
|
|
|
|
positions of writes and flushes to the publisher, and when using
|
|
|
|
|
synchronous replication, the publisher will wait for the actual
|
|
|
|
|
flush. This means that setting
|
|
|
|
|
<literal>synchronous_commit</literal> for the subscriber to
|
|
|
|
|
<literal>off</literal> when the subscription is used for
|
|
|
|
|
synchronous replication might increase the latency for
|
|
|
|
|
<command>COMMIT</command> on the publisher. In this scenario, it
|
|
|
|
|
can be advantageous to set <literal>synchronous_commit</literal>
|
|
|
|
|
to <literal>local</literal> or higher.
|
|
|
|
|
</para>
|
Add support for prepared transactions to built-in logical replication.
To add support for streaming transactions at prepare time into the
built-in logical replication, we need to do the following things:
* Modify the output plugin (pgoutput) to implement the new two-phase API
callbacks, by leveraging the extended replication protocol.
* Modify the replication apply worker, to properly handle two-phase
transactions by replaying them on prepare.
* Add a new SUBSCRIPTION option "two_phase" to allow users to enable
two-phase transactions. We enable the two_phase once the initial data sync
is over.
We however must explicitly disable replication of two-phase transactions
during replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover, we don't have a replication connection open so we don't know
where to send the data anyway.
The streaming option is not allowed with this new two_phase option. This
can be done as a separate patch.
We don't allow to toggle two_phase option of a subscription because it can
lead to an inconsistent replica. For the same reason, we don't allow to
refresh the publication once the two_phase is enabled for a subscription
unless copy_data option is false.
Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
2021-07-14 04:03:50 +02:00
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
2021-09-13 04:59:10 +02:00
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-two-phase">
|
Add support for prepared transactions to built-in logical replication.
To add support for streaming transactions at prepare time into the
built-in logical replication, we need to do the following things:
* Modify the output plugin (pgoutput) to implement the new two-phase API
callbacks, by leveraging the extended replication protocol.
* Modify the replication apply worker, to properly handle two-phase
transactions by replaying them on prepare.
* Add a new SUBSCRIPTION option "two_phase" to allow users to enable
two-phase transactions. We enable the two_phase once the initial data sync
is over.
We however must explicitly disable replication of two-phase transactions
during replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover, we don't have a replication connection open so we don't know
where to send the data anyway.
The streaming option is not allowed with this new two_phase option. This
can be done as a separate patch.
We don't allow to toggle two_phase option of a subscription because it can
lead to an inconsistent replica. For the same reason, we don't allow to
refresh the publication once the two_phase is enabled for a subscription
unless copy_data option is false.
Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
2021-07-14 04:03:50 +02:00
|
|
|
<term><literal>two_phase</literal> (<type>boolean</type>)</term>
|
|
|
|
|
<listitem>
|
|
|
|
|
<para>
|
|
|
|
|
Specifies whether two-phase commit is enabled for this subscription.
|
|
|
|
|
The default is <literal>false</literal>.
|
|
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
When two-phase commit is enabled, prepared transactions are sent
|
|
|
|
|
to the subscriber at the time of <command>PREPARE
|
|
|
|
|
TRANSACTION</command>, and are processed as two-phase
|
|
|
|
|
transactions on the subscriber too. Otherwise, prepared
|
|
|
|
|
transactions are sent to the subscriber only when committed, and
|
|
|
|
|
are then processed immediately by the subscriber.
|
Add support for prepared transactions to built-in logical replication.
To add support for streaming transactions at prepare time into the
built-in logical replication, we need to do the following things:
* Modify the output plugin (pgoutput) to implement the new two-phase API
callbacks, by leveraging the extended replication protocol.
* Modify the replication apply worker, to properly handle two-phase
transactions by replaying them on prepare.
* Add a new SUBSCRIPTION option "two_phase" to allow users to enable
two-phase transactions. We enable the two_phase once the initial data sync
is over.
We however must explicitly disable replication of two-phase transactions
during replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover, we don't have a replication connection open so we don't know
where to send the data anyway.
The streaming option is not allowed with this new two_phase option. This
can be done as a separate patch.
We don't allow to toggle two_phase option of a subscription because it can
lead to an inconsistent replica. For the same reason, we don't allow to
refresh the publication once the two_phase is enabled for a subscription
unless copy_data option is false.
Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
2021-07-14 04:03:50 +02:00
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
2021-09-13 20:27:02 +02:00
|
|
|
The implementation of two-phase commit requires that replication
|
|
|
|
|
has successfully finished the initial table synchronization
|
|
|
|
|
phase. So even when <literal>two_phase</literal> is enabled for a
|
|
|
|
|
subscription, the internal two-phase state remains
|
|
|
|
|
temporarily <quote>pending</quote> until the initialization phase
|
|
|
|
|
completes. See column <structfield>subtwophasestate</structfield>
|
|
|
|
|
of <link linkend="catalog-pg-subscription"><structname>pg_subscription</structname></link>
|
Add support for prepared transactions to built-in logical replication.
To add support for streaming transactions at prepare time into the
built-in logical replication, we need to do the following things:
* Modify the output plugin (pgoutput) to implement the new two-phase API
callbacks, by leveraging the extended replication protocol.
* Modify the replication apply worker, to properly handle two-phase
transactions by replaying them on prepare.
* Add a new SUBSCRIPTION option "two_phase" to allow users to enable
two-phase transactions. We enable the two_phase once the initial data sync
is over.
We however must explicitly disable replication of two-phase transactions
during replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover, we don't have a replication connection open so we don't know
where to send the data anyway.
The streaming option is not allowed with this new two_phase option. This
can be done as a separate patch.
We don't allow to toggle two_phase option of a subscription because it can
lead to an inconsistent replica. For the same reason, we don't allow to
refresh the publication once the two_phase is enabled for a subscription
unless copy_data option is false.
Author: Peter Smith, Ajin Cherian and Amit Kapila based on previous work by Nikhil Sontakke and Stas Kelvich
Reviewed-by: Amit Kapila, Sawada Masahiko, Vignesh C, Dilip Kumar, Takamichi Osumi, Greg Nancarrow
Tested-By: Haiying Tang
Discussion: https://postgr.es/m/02DA5F5E-CECE-4D9C-8B4B-418077E2C010@postgrespro.ru
Discussion: https://postgr.es/m/CAA4eK1+opiV4aFTmWWUF9h_32=HfPOW9vZASHarT0UA5oBrtGw@mail.gmail.com
2021-07-14 04:03:50 +02:00
|
|
|
to know the actual two-phase state.
|
|
|
|
|
</para>
|
|
|
|
|
|
Add support for streaming to built-in logical replication.
To add support for streaming of in-progress transactions into the
built-in logical replication, we need to do three things:
* Extend the logical replication protocol, so identify in-progress
transactions, and allow adding additional bits of information (e.g.
XID of subtransactions).
* Modify the output plugin (pgoutput) to implement the new stream
API callbacks, by leveraging the extended replication protocol.
* Modify the replication apply worker, to properly handle streamed
in-progress transaction by spilling the data to disk and then
replaying them on commit.
We however must explicitly disable streaming replication during
replication slot creation, even if the plugin supports it. We
don't need to replicate the changes accumulated during this phase,
and moreover we don't have a replication connection open so we
don't have where to send the data anyway.
Author: Tomas Vondra, Dilip Kumar and Amit Kapila
Reviewed-by: Amit Kapila, Kuntal Ghosh and Ajin Cherian
Tested-by: Neha Sharma, Mahendra Singh Thalor and Ajin Cherian
Discussion: https://postgr.es/m/688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com
2020-09-03 04:24:07 +02:00
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
Optionally disable subscriptions on error.
Logical replication apply workers for a subscription can easily get stuck
in an infinite loop of attempting to apply a change, triggering an error
(such as a constraint violation), exiting with the error written to the
subscription server log, and restarting.
To partially remedy the situation, this patch adds a new subscription
option named 'disable_on_error'. To be consistent with old behavior, this
option defaults to false. When true, both the tablesync worker and apply
worker catch any errors thrown and disable the subscription in order to
break the loop. The error is still also written in the logs.
Once the subscription is disabled, users can either manually resolve the
conflict/error or skip the conflicting transaction by using
pg_replication_origin_advance() function. After resolving the conflict,
users need to enable the subscription to allow apply process to proceed.
Author: Osumi Takamichi and Mark Dilger
Reviewed-by: Greg Nancarrow, Vignesh C, Amit Kapila, Wang wei, Tang Haiying, Peter Smith, Masahiko Sawada, Shi Yu
Discussion : https://postgr.es/m/DB35438F-9356-4841-89A0-412709EBD3AB%40enterprisedb.com
2022-03-14 05:02:40 +01:00
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-disable-on-error">
|
Optionally disable subscriptions on error.
Logical replication apply workers for a subscription can easily get stuck
in an infinite loop of attempting to apply a change, triggering an error
(such as a constraint violation), exiting with the error written to the
subscription server log, and restarting.
To partially remedy the situation, this patch adds a new subscription
option named 'disable_on_error'. To be consistent with old behavior, this
option defaults to false. When true, both the tablesync worker and apply
worker catch any errors thrown and disable the subscription in order to
break the loop. The error is still also written in the logs.
Once the subscription is disabled, users can either manually resolve the
conflict/error or skip the conflicting transaction by using
pg_replication_origin_advance() function. After resolving the conflict,
users need to enable the subscription to allow apply process to proceed.
Author: Osumi Takamichi and Mark Dilger
Reviewed-by: Greg Nancarrow, Vignesh C, Amit Kapila, Wang wei, Tang Haiying, Peter Smith, Masahiko Sawada, Shi Yu
Discussion : https://postgr.es/m/DB35438F-9356-4841-89A0-412709EBD3AB%40enterprisedb.com
2022-03-14 05:02:40 +01:00
|
|
|
<term><literal>disable_on_error</literal> (<type>boolean</type>)</term>
|
|
|
|
|
<listitem>
|
|
|
|
|
<para>
|
|
|
|
|
Specifies whether the subscription should be automatically disabled
|
|
|
|
|
if any errors are detected by subscription workers during data
|
|
|
|
|
replication from the publisher. The default is
|
|
|
|
|
<literal>false</literal>.
|
|
|
|
|
</para>
|
|
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
2022-07-21 05:17:38 +02:00
|
|
|
|
2023-03-30 18:06:34 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-password-required">
|
2023-04-08 06:12:50 +02:00
|
|
|
<term><literal>password_required</literal> (<type>boolean</type>)</term>
|
2023-03-30 18:06:34 +02:00
|
|
|
<listitem>
|
|
|
|
|
<para>
|
|
|
|
|
Specifies whether connections to the publisher made as a result
|
|
|
|
|
of this subscription must use password authentication. This setting
|
|
|
|
|
is ignored when the subscription is owned by a superuser.
|
|
|
|
|
The default is <literal>true</literal>. Only superusers can set
|
|
|
|
|
this value to <literal>false</literal>.
|
|
|
|
|
</para>
|
|
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
|
|
|
|
|
2023-04-04 18:03:03 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-run-as-owner">
|
2023-04-08 06:12:50 +02:00
|
|
|
<term><literal>run_as_owner</literal> (<type>boolean</type>)</term>
|
2023-04-04 18:03:03 +02:00
|
|
|
<listitem>
|
|
|
|
|
<para>
|
|
|
|
|
If true, all replication actions are performed as the subscription
|
|
|
|
|
owner. If false, replication workers will perform actions on each
|
|
|
|
|
table as the owner of that table. The latter configuration is
|
|
|
|
|
generally much more secure; for details, see
|
|
|
|
|
<xref linkend="logical-replication-security" />.
|
|
|
|
|
The default is <literal>false</literal>.
|
|
|
|
|
</para>
|
|
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
|
|
|
|
|
2023-03-29 06:28:14 +02:00
|
|
|
<varlistentry id="sql-createsubscription-with-origin">
|
2022-07-21 05:17:38 +02:00
|
|
|
<term><literal>origin</literal> (<type>string</type>)</term>
|
|
|
|
|
<listitem>
|
|
|
|
|
<para>
|
|
|
|
|
Specifies whether the subscription will request the publisher to only
|
|
|
|
|
send changes that don't have an origin or send changes regardless of
|
|
|
|
|
origin. Setting <literal>origin</literal> to <literal>none</literal>
|
|
|
|
|
means that the subscription will request the publisher to only send
|
|
|
|
|
changes that don't have an origin. Setting <literal>origin</literal>
|
|
|
|
|
to <literal>any</literal> means that the publisher sends changes
|
|
|
|
|
regardless of their origin. The default is <literal>any</literal>.
|
|
|
|
|
</para>
|
2022-09-08 03:24:13 +02:00
|
|
|
<para>
|
|
|
|
|
See <xref linkend="sql-createsubscription-notes"/> for details of how
|
|
|
|
|
<literal>copy_data = true</literal> can interact with the
|
|
|
|
|
<literal>origin</literal> parameter.
|
|
|
|
|
</para>
|
2022-07-21 05:17:38 +02:00
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
2022-04-07 18:23:28 +02:00
|
|
|
</variablelist></para>
|
2021-09-13 04:59:10 +02:00
|
|
|
|
2017-03-23 13:36:36 +01:00
|
|
|
</listitem>
|
|
|
|
|
</varlistentry>
|
2017-01-19 18:00:00 +01:00
|
|
|
</variablelist>
|
2023-01-30 17:59:37 +01:00
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
When specifying a parameter of type <type>boolean</type>, the
|
|
|
|
|
<literal>=</literal> <replaceable class="parameter">value</replaceable>
|
|
|
|
|
part can be omitted, which is equivalent to
|
|
|
|
|
specifying <literal>TRUE</literal>.
|
|
|
|
|
</para>
|
2017-01-19 18:00:00 +01:00
|
|
|
</refsect1>
|
|
|
|
|
|
Allow specifying row filters for logical replication of tables.
This feature adds row filtering for publication tables. When a publication
is defined or modified, an optional WHERE clause can be specified. Rows
that don't satisfy this WHERE clause will be filtered out. This allows a
set of tables to be partially replicated. The row filter is per table. A
new row filter can be added simply by specifying a WHERE clause after the
table name. The WHERE clause must be enclosed by parentheses.
The row filter WHERE clause for a table added to a publication that
publishes UPDATE and/or DELETE operations must contain only columns that
are covered by REPLICA IDENTITY. The row filter WHERE clause for a table
added to a publication that publishes INSERT can use any column. If the
row filter evaluates to NULL, it is regarded as "false". The WHERE clause
only allows simple expressions that don't have user-defined functions,
user-defined operators, user-defined types, user-defined collations,
non-immutable built-in functions, or references to system columns. These
restrictions could be addressed in the future.
If you choose to do the initial table synchronization, only data that
satisfies the row filters is copied to the subscriber. If the subscription
has several publications in which a table has been published with
different WHERE clauses, rows that satisfy ANY of the expressions will be
copied. If a subscriber is a pre-15 version, the initial table
synchronization won't use row filters even if they are defined in the
publisher.
The row filters are applied before publishing the changes. If the
subscription has several publications in which the same table has been
published with different filters (for the same publish operation), those
expressions get OR'ed together so that rows satisfying any of the
expressions will be replicated.
This means all the other filters become redundant if (a) one of the
publications have no filter at all, (b) one of the publications was
created using FOR ALL TABLES, (c) one of the publications was created
using FOR ALL TABLES IN SCHEMA and the table belongs to that same schema.
If your publication contains a partitioned table, the publication
parameter publish_via_partition_root determines if it uses the partition's
row filter (if the parameter is false, the default) or the root
partitioned table's row filter.
Psql commands \dRp+ and \d <table-name> will display any row filters.
Author: Hou Zhijie, Euler Taveira, Peter Smith, Ajin Cherian
Reviewed-by: Greg Nancarrow, Haiying Tang, Amit Kapila, Tomas Vondra, Dilip Kumar, Vignesh C, Alvaro Herrera, Andres Freund, Wei Wang
Discussion: https://www.postgresql.org/message-id/flat/CAHE3wggb715X%2BmK_DitLXF25B%3DjE6xyNCH4YOwM860JR7HarGQ%40mail.gmail.com
2022-02-22 03:24:12 +01:00
|
|
|
<refsect1 id="sql-createsubscription-notes" xreflabel="Notes">
|
2017-02-08 03:26:50 +01:00
|
|
|
<title>Notes</title>
|
|
|
|
|
|
|
|
|
|
<para>
|
2017-11-23 15:39:47 +01:00
|
|
|
See <xref linkend="logical-replication-security"/> for details on
|
2017-02-08 03:26:50 +01:00
|
|
|
how to configure access control between the subscription and the
|
|
|
|
|
publication instance.
|
|
|
|
|
</para>
|
2017-06-09 23:11:46 +02:00
|
|
|
|
2017-09-22 21:01:13 +02:00
|
|
|
<para>
|
|
|
|
|
When creating a replication slot (the default behavior), <command>CREATE
|
|
|
|
|
SUBSCRIPTION</command> cannot be executed inside a transaction block.
|
|
|
|
|
</para>
|
|
|
|
|
|
2017-06-09 23:11:46 +02:00
|
|
|
<para>
|
|
|
|
|
Creating a subscription that connects to the same database cluster (for
|
|
|
|
|
example, to replicate between databases in the same cluster or to replicate
|
|
|
|
|
within the same database) will only succeed if the replication slot is not
|
|
|
|
|
created as part of the same command. Otherwise, the <command>CREATE
|
|
|
|
|
SUBSCRIPTION</command> call will hang. To make this work, create the
|
|
|
|
|
replication slot separately (using the
|
|
|
|
|
function <function>pg_create_logical_replication_slot</function> with the
|
|
|
|
|
plugin name <literal>pgoutput</literal>) and create the subscription using
|
2022-11-02 07:10:37 +01:00
|
|
|
the parameter <literal>create_slot = false</literal>. See
|
|
|
|
|
<xref linkend="logical-replication-subscription-examples-deferred-slot"/>
|
|
|
|
|
for examples. This is an implementation restriction that might be lifted in a
|
|
|
|
|
future release.
|
2017-06-09 23:11:46 +02:00
|
|
|
</para>
|
Allow specifying row filters for logical replication of tables.
This feature adds row filtering for publication tables. When a publication
is defined or modified, an optional WHERE clause can be specified. Rows
that don't satisfy this WHERE clause will be filtered out. This allows a
set of tables to be partially replicated. The row filter is per table. A
new row filter can be added simply by specifying a WHERE clause after the
table name. The WHERE clause must be enclosed by parentheses.
The row filter WHERE clause for a table added to a publication that
publishes UPDATE and/or DELETE operations must contain only columns that
are covered by REPLICA IDENTITY. The row filter WHERE clause for a table
added to a publication that publishes INSERT can use any column. If the
row filter evaluates to NULL, it is regarded as "false". The WHERE clause
only allows simple expressions that don't have user-defined functions,
user-defined operators, user-defined types, user-defined collations,
non-immutable built-in functions, or references to system columns. These
restrictions could be addressed in the future.
If you choose to do the initial table synchronization, only data that
satisfies the row filters is copied to the subscriber. If the subscription
has several publications in which a table has been published with
different WHERE clauses, rows that satisfy ANY of the expressions will be
copied. If a subscriber is a pre-15 version, the initial table
synchronization won't use row filters even if they are defined in the
publisher.
The row filters are applied before publishing the changes. If the
subscription has several publications in which the same table has been
published with different filters (for the same publish operation), those
expressions get OR'ed together so that rows satisfying any of the
expressions will be replicated.
This means all the other filters become redundant if (a) one of the
publications have no filter at all, (b) one of the publications was
created using FOR ALL TABLES, (c) one of the publications was created
using FOR ALL TABLES IN SCHEMA and the table belongs to that same schema.
If your publication contains a partitioned table, the publication
parameter publish_via_partition_root determines if it uses the partition's
row filter (if the parameter is false, the default) or the root
partitioned table's row filter.
Psql commands \dRp+ and \d <table-name> will display any row filters.
Author: Hou Zhijie, Euler Taveira, Peter Smith, Ajin Cherian
Reviewed-by: Greg Nancarrow, Haiying Tang, Amit Kapila, Tomas Vondra, Dilip Kumar, Vignesh C, Alvaro Herrera, Andres Freund, Wei Wang
Discussion: https://www.postgresql.org/message-id/flat/CAHE3wggb715X%2BmK_DitLXF25B%3DjE6xyNCH4YOwM860JR7HarGQ%40mail.gmail.com
2022-02-22 03:24:12 +01:00
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
If any table in the publication has a <literal>WHERE</literal> clause, rows
|
|
|
|
|
for which the <replaceable class="parameter">expression</replaceable>
|
|
|
|
|
evaluates to false or null will not be published. If the subscription has
|
|
|
|
|
several publications in which the same table has been published with
|
|
|
|
|
different <literal>WHERE</literal> clauses, a row will be published if any
|
|
|
|
|
of the expressions (referring to that publish operation) are satisfied. In
|
|
|
|
|
the case of different <literal>WHERE</literal> clauses, if one of the
|
|
|
|
|
publications has no <literal>WHERE</literal> clause (referring to that
|
|
|
|
|
publish operation) or the publication is declared as
|
2023-03-31 05:29:55 +02:00
|
|
|
<link linkend="sql-createpublication-for-all-tables"><literal>FOR ALL TABLES</literal></link>
|
|
|
|
|
or <link linkend="sql-createpublication-for-tables-in-schema"><literal>FOR TABLES IN SCHEMA</literal></link>,
|
|
|
|
|
rows are always published regardless of the definition of the other
|
|
|
|
|
expressions. If the subscriber is a <productname>PostgreSQL</productname>
|
|
|
|
|
version before 15, then any row filtering is ignored during the initial data
|
|
|
|
|
synchronization phase. For this case, the user might want to consider
|
|
|
|
|
deleting any initially copied data that would be incompatible with
|
|
|
|
|
subsequent filtering. Because initial data synchronization does not take
|
|
|
|
|
into account the publication
|
|
|
|
|
<link linkend="sql-createpublication-with-publish"><literal>publish</literal></link>
|
|
|
|
|
parameter when copying existing table data, some rows may be copied that
|
|
|
|
|
would not be replicated using DML. See
|
2022-06-24 05:07:26 +02:00
|
|
|
<xref linkend="logical-replication-subscription-examples"/> for examples.
|
Allow specifying row filters for logical replication of tables.
This feature adds row filtering for publication tables. When a publication
is defined or modified, an optional WHERE clause can be specified. Rows
that don't satisfy this WHERE clause will be filtered out. This allows a
set of tables to be partially replicated. The row filter is per table. A
new row filter can be added simply by specifying a WHERE clause after the
table name. The WHERE clause must be enclosed by parentheses.
The row filter WHERE clause for a table added to a publication that
publishes UPDATE and/or DELETE operations must contain only columns that
are covered by REPLICA IDENTITY. The row filter WHERE clause for a table
added to a publication that publishes INSERT can use any column. If the
row filter evaluates to NULL, it is regarded as "false". The WHERE clause
only allows simple expressions that don't have user-defined functions,
user-defined operators, user-defined types, user-defined collations,
non-immutable built-in functions, or references to system columns. These
restrictions could be addressed in the future.
If you choose to do the initial table synchronization, only data that
satisfies the row filters is copied to the subscriber. If the subscription
has several publications in which a table has been published with
different WHERE clauses, rows that satisfy ANY of the expressions will be
copied. If a subscriber is a pre-15 version, the initial table
synchronization won't use row filters even if they are defined in the
publisher.
The row filters are applied before publishing the changes. If the
subscription has several publications in which the same table has been
published with different filters (for the same publish operation), those
expressions get OR'ed together so that rows satisfying any of the
expressions will be replicated.
This means all the other filters become redundant if (a) one of the
publications have no filter at all, (b) one of the publications was
created using FOR ALL TABLES, (c) one of the publications was created
using FOR ALL TABLES IN SCHEMA and the table belongs to that same schema.
If your publication contains a partitioned table, the publication
parameter publish_via_partition_root determines if it uses the partition's
row filter (if the parameter is false, the default) or the root
partitioned table's row filter.
Psql commands \dRp+ and \d <table-name> will display any row filters.
Author: Hou Zhijie, Euler Taveira, Peter Smith, Ajin Cherian
Reviewed-by: Greg Nancarrow, Haiying Tang, Amit Kapila, Tomas Vondra, Dilip Kumar, Vignesh C, Alvaro Herrera, Andres Freund, Wei Wang
Discussion: https://www.postgresql.org/message-id/flat/CAHE3wggb715X%2BmK_DitLXF25B%3DjE6xyNCH4YOwM860JR7HarGQ%40mail.gmail.com
2022-02-22 03:24:12 +01:00
|
|
|
</para>
|
|
|
|
|
|
2022-06-02 05:01:50 +02:00
|
|
|
<para>
|
|
|
|
|
Subscriptions having several publications in which the same table has been
|
|
|
|
|
published with different column lists are not supported.
|
|
|
|
|
</para>
|
|
|
|
|
|
2022-03-31 04:54:19 +02:00
|
|
|
<para>
|
|
|
|
|
We allow non-existent publications to be specified so that users can add
|
|
|
|
|
those later. This means
|
|
|
|
|
<link linkend="catalog-pg-subscription"><structname>pg_subscription</structname></link>
|
|
|
|
|
can have non-existent publications.
|
|
|
|
|
</para>
|
|
|
|
|
|
2022-09-08 03:24:13 +02:00
|
|
|
<para>
|
|
|
|
|
When using a subscription parameter combination of
|
|
|
|
|
<literal>copy_data = true</literal> and <literal>origin = NONE</literal>,
|
|
|
|
|
the initial sync table data is copied directly from the publisher, meaning
|
|
|
|
|
that knowledge of the true origin of that data is not possible. If the
|
|
|
|
|
publisher also has subscriptions then the copied table data might have
|
|
|
|
|
originated from further upstream. This scenario is detected and a WARNING is
|
|
|
|
|
logged to the user, but the warning is only an indication of a potential
|
|
|
|
|
problem; it is the user's responsibility to make the necessary checks to
|
|
|
|
|
ensure the copied data origins are really as wanted or not.
|
|
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
To find which tables might potentially include non-local origins (due to
|
|
|
|
|
other subscriptions created on the publisher) try this SQL query:
|
|
|
|
|
<programlisting>
|
|
|
|
|
# substitute <pub-names> below with your publication name(s) to be queried
|
2023-04-10 18:22:10 +02:00
|
|
|
SELECT DISTINCT PT.schemaname, PT.tablename
|
|
|
|
|
FROM pg_publication_tables PT,
|
|
|
|
|
pg_subscription_rel PS
|
|
|
|
|
JOIN pg_class C ON (C.oid = PS.srrelid)
|
|
|
|
|
JOIN pg_namespace N ON (N.oid = C.relnamespace)
|
|
|
|
|
WHERE N.nspname = PT.schemaname AND
|
|
|
|
|
C.relname = PT.tablename AND
|
|
|
|
|
PT.pubname IN (<pub-names>);
|
2022-09-08 03:24:13 +02:00
|
|
|
</programlisting></para>
|
|
|
|
|
|
2017-02-08 03:26:50 +01:00
|
|
|
</refsect1>
|
|
|
|
|
|
2017-01-19 18:00:00 +01:00
|
|
|
<refsect1>
|
|
|
|
|
<title>Examples</title>
|
|
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
Create a subscription to a remote server that replicates tables in
|
2017-02-18 00:59:29 +01:00
|
|
|
the publications <literal>mypublication</literal> and
|
2017-01-19 18:00:00 +01:00
|
|
|
<literal>insert_only</literal> and starts replicating immediately on
|
|
|
|
|
commit:
|
|
|
|
|
<programlisting>
|
|
|
|
|
CREATE SUBSCRIPTION mysub
|
2017-02-08 03:26:50 +01:00
|
|
|
CONNECTION 'host=192.168.1.50 port=5432 user=foo dbname=foodb'
|
2017-01-19 18:00:00 +01:00
|
|
|
PUBLICATION mypublication, insert_only;
|
|
|
|
|
</programlisting>
|
|
|
|
|
</para>
|
|
|
|
|
|
|
|
|
|
<para>
|
|
|
|
|
Create a subscription to a remote server that replicates tables in
|
|
|
|
|
the <literal>insert_only</literal> publication and does not start replicating
|
|
|
|
|
until enabled at a later time.
|
|
|
|
|
<programlisting>
|
|
|
|
|
CREATE SUBSCRIPTION mysub
|
2017-02-08 03:26:50 +01:00
|
|
|
CONNECTION 'host=192.168.1.50 port=5432 user=foo dbname=foodb'
|
2017-01-19 18:00:00 +01:00
|
|
|
PUBLICATION insert_only
|
2017-05-12 14:57:01 +02:00
|
|
|
WITH (enabled = false);
|
2017-06-14 19:55:43 +02:00
|
|
|
</programlisting></para>
|
2017-01-19 18:00:00 +01:00
|
|
|
</refsect1>
|
|
|
|
|
|
|
|
|
|
<refsect1>
|
|
|
|
|
<title>Compatibility</title>
|
|
|
|
|
|
|
|
|
|
<para>
|
2017-10-09 03:44:17 +02:00
|
|
|
<command>CREATE SUBSCRIPTION</command> is a <productname>PostgreSQL</productname>
|
2017-01-19 18:00:00 +01:00
|
|
|
extension.
|
|
|
|
|
</para>
|
|
|
|
|
</refsect1>
|
|
|
|
|
|
|
|
|
|
<refsect1>
|
|
|
|
|
<title>See Also</title>
|
|
|
|
|
|
|
|
|
|
<simplelist type="inline">
|
2017-11-23 15:39:47 +01:00
|
|
|
<member><xref linkend="sql-altersubscription"/></member>
|
|
|
|
|
<member><xref linkend="sql-dropsubscription"/></member>
|
|
|
|
|
<member><xref linkend="sql-createpublication"/></member>
|
|
|
|
|
<member><xref linkend="sql-alterpublication"/></member>
|
2017-01-19 18:00:00 +01:00
|
|
|
</simplelist>
|
|
|
|
|
</refsect1>
|
|
|
|
|
</refentry>
|
