2013-02-21 11:26:23 +01:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
*
|
|
|
|
* postgres_fdw.h
|
|
|
|
* Foreign-data wrapper for remote PostgreSQL servers
|
|
|
|
*
|
2024-01-04 02:49:05 +01:00
|
|
|
* Portions Copyright (c) 2012-2024, PostgreSQL Global Development Group
|
2013-02-21 11:26:23 +01:00
|
|
|
*
|
|
|
|
* IDENTIFICATION
|
|
|
|
* contrib/postgres_fdw/postgres_fdw.h
|
|
|
|
*
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
*/
|
|
|
|
#ifndef POSTGRES_FDW_H
|
|
|
|
#define POSTGRES_FDW_H
|
|
|
|
|
|
|
|
#include "foreign/foreign.h"
|
|
|
|
#include "lib/stringinfo.h"
|
2019-11-25 03:38:57 +01:00
|
|
|
#include "libpq-fe.h"
|
Add support for asynchronous execution.
This implements asynchronous execution, which runs multiple parts of a
non-parallel-aware Append concurrently rather than serially to improve
performance when possible. Currently, the only node type that can be
run concurrently is a ForeignScan that is an immediate child of such an
Append. In the case where such ForeignScans access data on different
remote servers, this would run those ForeignScans concurrently, and
overlap the remote operations to be performed simultaneously, so it'll
improve the performance especially when the operations involve
time-consuming ones such as remote join and remote aggregation.
We may extend this to other node types such as joins or aggregates over
ForeignScans in the future.
This also adds the support for postgres_fdw, which is enabled by the
table-level/server-level option "async_capable". The default is false.
Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit
is mostly based on the patch proposed by Robert Haas, but also uses
stuff from the patch proposed by Kyotaro Horiguchi and from the patch
proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin
Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and
others.
Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com
Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
2021-03-31 11:45:00 +02:00
|
|
|
#include "nodes/execnodes.h"
|
2019-01-29 22:49:25 +01:00
|
|
|
#include "nodes/pathnodes.h"
|
2014-11-15 01:48:53 +01:00
|
|
|
#include "utils/relcache.h"
|
2013-02-21 11:26:23 +01:00
|
|
|
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
/*
|
|
|
|
* FDW-specific planner information kept in RelOptInfo.fdw_private for a
|
2017-04-11 19:53:13 +02:00
|
|
|
* postgres_fdw foreign table. For a baserel, this struct is created by
|
|
|
|
* postgresGetForeignRelSize, although some fields are not filled till later.
|
|
|
|
* postgresGetForeignJoinPaths creates it for a joinrel, and
|
|
|
|
* postgresGetForeignUpperPaths creates it for an upperrel.
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
*/
|
|
|
|
typedef struct PgFdwRelationInfo
|
|
|
|
{
|
2016-02-09 20:00:50 +01:00
|
|
|
/*
|
|
|
|
* True means that the relation can be pushed down. Always true for simple
|
|
|
|
* foreign scan.
|
|
|
|
*/
|
|
|
|
bool pushdown_safe;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Restriction clauses, divided into safe and unsafe to pushdown subsets.
|
Handle restriction clause lists more uniformly in postgres_fdw.
Clauses in the lists retained by postgres_fdw during planning were
sometimes bare boolean clauses, sometimes RestrictInfos, and sometimes
a mixture of the two in the same list. The comment about that situation
didn't come close to telling the full truth, either. Aside from being
confusing, this had a couple of bad practical consequences:
* waste of planning cycles due to inability to cache per-clause selectivity
and cost estimates;
* sometimes, RestrictInfos would sneak into the fdw_private list of a
finished Plan node, causing failures if, for example, we tried to ship
the Plan tree to a parallel worker.
(It may well be that it's a bug in the parallel-query logic that we
would ever try to ship such a plan to a parallel worker, but in any
case this deserves to be cleaned up.)
To fix, rearrange so that clause lists in PgFdwRelationInfo are always
lists of RestrictInfos, and then strip the RestrictInfos at the last
minute when making a Plan node. In passing do a bit of refactoring and
comment cleanup in postgresGetForeignPlan and foreign_join_ok.
Although the messiness here dates back at least to 9.6, there's no evidence
that it causes anything worse than wasted planning cycles in 9.6, so no
back-patch for now.
Per fuzz testing by Andreas Seltenreich.
Tom Lane and Ashutosh Bapat
Discussion: https://postgr.es/m/87tw5x4vcu.fsf@credativ.de
2017-04-11 17:58:59 +02:00
|
|
|
* All entries in these lists should have RestrictInfo wrappers; that
|
|
|
|
* improves efficiency of selectivity and cost estimation.
|
2016-02-09 20:00:50 +01:00
|
|
|
*/
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
List *remote_conds;
|
|
|
|
List *local_conds;
|
|
|
|
|
2017-04-11 19:53:13 +02:00
|
|
|
/* Actual remote restriction clauses for scan (sans RestrictInfos) */
|
|
|
|
List *final_remote_exprs;
|
|
|
|
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
/* Bitmap of attr numbers we need to fetch from the remote server. */
|
|
|
|
Bitmapset *attrs_used;
|
|
|
|
|
2019-04-02 12:20:30 +02:00
|
|
|
/* True means that the query_pathkeys is safe to push down */
|
|
|
|
bool qp_is_pushdown_safe;
|
|
|
|
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
/* Cost and selectivity of local_conds. */
|
|
|
|
QualCost local_conds_cost;
|
|
|
|
Selectivity local_conds_sel;
|
|
|
|
|
2016-02-09 20:00:50 +01:00
|
|
|
/* Selectivity of join conditions */
|
|
|
|
Selectivity joinclause_sel;
|
|
|
|
|
postgres_fdw: Fix costing of pre-sorted foreign paths with local stats.
Commit aa09cd242 modified estimate_path_cost_size() so that it reuses
cached costs of a basic foreign path for a given foreign-base/join
relation when costing pre-sorted foreign paths for that relation, but it
incorrectly re-computed retrieved_rows, an estimated number of rows
fetched from the remote side, which is needed for costing both the basic
and pre-sorted foreign paths. To fix, handle retrieved_rows the same way
as the cached costs: store in that relation's fpinfo the retrieved_rows
estimate computed for costing the basic foreign path, and reuse it when
costing the pre-sorted foreign paths. Also, reuse the rows/width
estimates stored in that relation's fpinfo when costing the pre-sorted
foreign paths, to make the code consistent.
In commit ffab494a4, to extend the costing mentioned above to the
foreign-grouping case, I made a change to add_foreign_grouping_paths() to
store in a given foreign-grouped relation's RelOptInfo the rows estimate
for that relation for reuse, but this patch makes that change unnecessary
since we already store the row estimate in that relation's fpinfo, which
this patch reuses when costing a foreign path for that relation with the
sortClause ordering; remove that change.
In passing, fix thinko in commit 7012b132d: in estimate_path_cost_size(),
the width estimate for a given foreign-grouped relation to be stored in
that relation's fpinfo was reset incorrectly when costing a basic foreign
path for that relation with local stats.
Apply the patch to HEAD only to avoid destabilizing existing plan choices.
Author: Etsuro Fujita
Discussion: https://postgr.es/m/CAPmGK17jaJLPDEkgnP2VmkOg=5wT8YQ1CqssU8JRpZ_NSE+dqQ@mail.gmail.com
2019-06-14 13:49:59 +02:00
|
|
|
/* Estimated size and cost for a scan, join, or grouping/aggregation. */
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
double rows;
|
|
|
|
int width;
|
|
|
|
Cost startup_cost;
|
|
|
|
Cost total_cost;
|
2019-07-01 18:37:52 +02:00
|
|
|
|
postgres_fdw: Fix costing of pre-sorted foreign paths with local stats.
Commit aa09cd242 modified estimate_path_cost_size() so that it reuses
cached costs of a basic foreign path for a given foreign-base/join
relation when costing pre-sorted foreign paths for that relation, but it
incorrectly re-computed retrieved_rows, an estimated number of rows
fetched from the remote side, which is needed for costing both the basic
and pre-sorted foreign paths. To fix, handle retrieved_rows the same way
as the cached costs: store in that relation's fpinfo the retrieved_rows
estimate computed for costing the basic foreign path, and reuse it when
costing the pre-sorted foreign paths. Also, reuse the rows/width
estimates stored in that relation's fpinfo when costing the pre-sorted
foreign paths, to make the code consistent.
In commit ffab494a4, to extend the costing mentioned above to the
foreign-grouping case, I made a change to add_foreign_grouping_paths() to
store in a given foreign-grouped relation's RelOptInfo the rows estimate
for that relation for reuse, but this patch makes that change unnecessary
since we already store the row estimate in that relation's fpinfo, which
this patch reuses when costing a foreign path for that relation with the
sortClause ordering; remove that change.
In passing, fix thinko in commit 7012b132d: in estimate_path_cost_size(),
the width estimate for a given foreign-grouped relation to be stored in
that relation's fpinfo was reset incorrectly when costing a basic foreign
path for that relation with local stats.
Apply the patch to HEAD only to avoid destabilizing existing plan choices.
Author: Etsuro Fujita
Discussion: https://postgr.es/m/CAPmGK17jaJLPDEkgnP2VmkOg=5wT8YQ1CqssU8JRpZ_NSE+dqQ@mail.gmail.com
2019-06-14 13:49:59 +02:00
|
|
|
/*
|
|
|
|
* Estimated number of rows fetched from the foreign server, and costs
|
|
|
|
* excluding costs for transferring those rows from the foreign server.
|
|
|
|
* These are only used by estimate_path_cost_size().
|
|
|
|
*/
|
|
|
|
double retrieved_rows;
|
2016-02-09 20:00:50 +01:00
|
|
|
Cost rel_startup_cost;
|
|
|
|
Cost rel_total_cost;
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
|
|
|
|
/* Options extracted from catalogs. */
|
|
|
|
bool use_remote_estimate;
|
|
|
|
Cost fdw_startup_cost;
|
|
|
|
Cost fdw_tuple_cost;
|
2021-01-05 02:00:16 +01:00
|
|
|
List *shippable_extensions; /* OIDs of shippable extensions */
|
Add support for asynchronous execution.
This implements asynchronous execution, which runs multiple parts of a
non-parallel-aware Append concurrently rather than serially to improve
performance when possible. Currently, the only node type that can be
run concurrently is a ForeignScan that is an immediate child of such an
Append. In the case where such ForeignScans access data on different
remote servers, this would run those ForeignScans concurrently, and
overlap the remote operations to be performed simultaneously, so it'll
improve the performance especially when the operations involve
time-consuming ones such as remote join and remote aggregation.
We may extend this to other node types such as joins or aggregates over
ForeignScans in the future.
This also adds the support for postgres_fdw, which is enabled by the
table-level/server-level option "async_capable". The default is false.
Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit
is mostly based on the patch proposed by Robert Haas, but also uses
stuff from the patch proposed by Kyotaro Horiguchi and from the patch
proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin
Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and
others.
Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com
Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
2021-03-31 11:45:00 +02:00
|
|
|
bool async_capable;
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
|
|
|
|
/* Cached catalog information. */
|
|
|
|
ForeignTable *table;
|
|
|
|
ForeignServer *server;
|
|
|
|
UserMapping *user; /* only set in use_remote_estimate mode */
|
2016-02-03 15:01:59 +01:00
|
|
|
|
2016-06-10 00:02:36 +02:00
|
|
|
int fetch_size; /* fetch size for this remote table */
|
2016-02-09 20:00:50 +01:00
|
|
|
|
|
|
|
/*
|
Make postgres_fdw's "Relations" output agree with the rest of EXPLAIN.
The relation aliases shown in the "Relations" line for a foreign scan
didn't always agree with those used in the rest of EXPLAIN's output.
The regression test result changes appearing here provide examples.
It's really impossible for postgres_fdw to duplicate EXPLAIN's alias
assignment logic during postgresGetForeignRelSize(), because of the
de-duplication that EXPLAIN does on a global basis --- and anyway,
trying to duplicate that would be unmaintainable. Instead, just put
numeric rangetable indexes into the string, and convert those to
table names/aliases in postgresExplainForeignScan, which does have
access to the results of ruleutils.c's alias assignment logic.
Aside from being more reliable, this shifts some work from planning
to EXPLAIN, which is a good tradeoff for performance. (I also
changed from using StringInfo to using psprintf, which makes the
code slightly simpler and reduces its memory consumption.)
A kluge required by this solution is that we have to reverse-engineer
the rtoffset applied by setrefs.c. If that logic ever fails
(presumably because the member tables of a join got offset by
different amounts), we'll need some more cooperation with setrefs.c
to keep things straight. But for now, there's no need for that.
Arguably this is a back-patchable bug fix, but since this is a mostly
cosmetic issue and there have been no field complaints, I'll refrain
for now.
Discussion: https://postgr.es/m/12424.1575168015@sss.pgh.pa.us
2019-12-02 22:31:03 +01:00
|
|
|
* Name of the relation, for use while EXPLAINing ForeignScan. It is used
|
|
|
|
* for join and upper relations but is set for all relations. For a base
|
|
|
|
* relation, this is really just the RT index as a string; we convert that
|
|
|
|
* while producing EXPLAIN output. For join and upper relations, the name
|
|
|
|
* indicates which base foreign tables are included and the join type or
|
|
|
|
* aggregation type used.
|
2016-02-09 20:00:50 +01:00
|
|
|
*/
|
Make postgres_fdw's "Relations" output agree with the rest of EXPLAIN.
The relation aliases shown in the "Relations" line for a foreign scan
didn't always agree with those used in the rest of EXPLAIN's output.
The regression test result changes appearing here provide examples.
It's really impossible for postgres_fdw to duplicate EXPLAIN's alias
assignment logic during postgresGetForeignRelSize(), because of the
de-duplication that EXPLAIN does on a global basis --- and anyway,
trying to duplicate that would be unmaintainable. Instead, just put
numeric rangetable indexes into the string, and convert those to
table names/aliases in postgresExplainForeignScan, which does have
access to the results of ruleutils.c's alias assignment logic.
Aside from being more reliable, this shifts some work from planning
to EXPLAIN, which is a good tradeoff for performance. (I also
changed from using StringInfo to using psprintf, which makes the
code slightly simpler and reduces its memory consumption.)
A kluge required by this solution is that we have to reverse-engineer
the rtoffset applied by setrefs.c. If that logic ever fails
(presumably because the member tables of a join got offset by
different amounts), we'll need some more cooperation with setrefs.c
to keep things straight. But for now, there's no need for that.
Arguably this is a back-patchable bug fix, but since this is a mostly
cosmetic issue and there have been no field complaints, I'll refrain
for now.
Discussion: https://postgr.es/m/12424.1575168015@sss.pgh.pa.us
2019-12-02 22:31:03 +01:00
|
|
|
char *relation_name;
|
2016-02-09 20:00:50 +01:00
|
|
|
|
|
|
|
/* Join information */
|
|
|
|
RelOptInfo *outerrel;
|
|
|
|
RelOptInfo *innerrel;
|
|
|
|
JoinType jointype;
|
2017-04-11 19:53:13 +02:00
|
|
|
/* joinclauses contains only JOIN/ON conditions for an outer join */
|
Handle restriction clause lists more uniformly in postgres_fdw.
Clauses in the lists retained by postgres_fdw during planning were
sometimes bare boolean clauses, sometimes RestrictInfos, and sometimes
a mixture of the two in the same list. The comment about that situation
didn't come close to telling the full truth, either. Aside from being
confusing, this had a couple of bad practical consequences:
* waste of planning cycles due to inability to cache per-clause selectivity
and cost estimates;
* sometimes, RestrictInfos would sneak into the fdw_private list of a
finished Plan node, causing failures if, for example, we tried to ship
the Plan tree to a parallel worker.
(It may well be that it's a bug in the parallel-query logic that we
would ever try to ship such a plan to a parallel worker, but in any
case this deserves to be cleaned up.)
To fix, rearrange so that clause lists in PgFdwRelationInfo are always
lists of RestrictInfos, and then strip the RestrictInfos at the last
minute when making a Plan node. In passing do a bit of refactoring and
comment cleanup in postgresGetForeignPlan and foreign_join_ok.
Although the messiness here dates back at least to 9.6, there's no evidence
that it causes anything worse than wasted planning cycles in 9.6, so no
back-patch for now.
Per fuzz testing by Andreas Seltenreich.
Tom Lane and Ashutosh Bapat
Discussion: https://postgr.es/m/87tw5x4vcu.fsf@credativ.de
2017-04-11 17:58:59 +02:00
|
|
|
List *joinclauses; /* List of RestrictInfo */
|
2016-10-21 15:54:29 +02:00
|
|
|
|
2019-04-02 12:20:30 +02:00
|
|
|
/* Upper relation information */
|
|
|
|
UpperRelationKind stage;
|
|
|
|
|
2016-10-21 15:54:29 +02:00
|
|
|
/* Grouping information */
|
|
|
|
List *grouped_tlist;
|
2017-03-16 18:34:59 +01:00
|
|
|
|
|
|
|
/* Subquery information */
|
2017-05-17 22:31:56 +02:00
|
|
|
bool make_outerrel_subquery; /* do we deparse outerrel as a
|
2017-03-16 18:34:59 +01:00
|
|
|
* subquery? */
|
2017-05-17 22:31:56 +02:00
|
|
|
bool make_innerrel_subquery; /* do we deparse innerrel as a
|
2017-03-16 18:34:59 +01:00
|
|
|
* subquery? */
|
|
|
|
Relids lower_subquery_rels; /* all relids appearing in lower
|
|
|
|
* subqueries */
|
2023-12-05 21:53:12 +01:00
|
|
|
Relids hidden_subquery_rels; /* relids, which can't be referred to
|
|
|
|
* from upper relations, used
|
|
|
|
* internally for equivalence member
|
|
|
|
* search */
|
2017-03-16 18:34:59 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Index of the relation. It is used to create an alias to a subquery
|
|
|
|
* representing the relation.
|
|
|
|
*/
|
|
|
|
int relation_index;
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
} PgFdwRelationInfo;
|
|
|
|
|
Add support for asynchronous execution.
This implements asynchronous execution, which runs multiple parts of a
non-parallel-aware Append concurrently rather than serially to improve
performance when possible. Currently, the only node type that can be
run concurrently is a ForeignScan that is an immediate child of such an
Append. In the case where such ForeignScans access data on different
remote servers, this would run those ForeignScans concurrently, and
overlap the remote operations to be performed simultaneously, so it'll
improve the performance especially when the operations involve
time-consuming ones such as remote join and remote aggregation.
We may extend this to other node types such as joins or aggregates over
ForeignScans in the future.
This also adds the support for postgres_fdw, which is enabled by the
table-level/server-level option "async_capable". The default is false.
Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit
is mostly based on the patch proposed by Robert Haas, but also uses
stuff from the patch proposed by Kyotaro Horiguchi and from the patch
proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin
Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and
others.
Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com
Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
2021-03-31 11:45:00 +02:00
|
|
|
/*
|
|
|
|
* Extra control information relating to a connection.
|
|
|
|
*/
|
|
|
|
typedef struct PgFdwConnState
|
|
|
|
{
|
|
|
|
AsyncRequest *pendingAreq; /* pending async request */
|
|
|
|
} PgFdwConnState;
|
|
|
|
|
2022-12-30 23:14:53 +01:00
|
|
|
/*
|
|
|
|
* Method used by ANALYZE to sample remote rows.
|
|
|
|
*/
|
|
|
|
typedef enum PgFdwSamplingMethod
|
|
|
|
{
|
|
|
|
ANALYZE_SAMPLE_OFF, /* no remote sampling */
|
|
|
|
ANALYZE_SAMPLE_AUTO, /* choose by server version */
|
|
|
|
ANALYZE_SAMPLE_RANDOM, /* remote random() */
|
|
|
|
ANALYZE_SAMPLE_SYSTEM, /* TABLESAMPLE system */
|
2023-10-26 09:16:25 +02:00
|
|
|
ANALYZE_SAMPLE_BERNOULLI, /* TABLESAMPLE bernoulli */
|
2022-12-30 23:14:53 +01:00
|
|
|
} PgFdwSamplingMethod;
|
|
|
|
|
2013-03-12 02:31:28 +01:00
|
|
|
/* in postgres_fdw.c */
|
|
|
|
extern int set_transmission_modes(void);
|
|
|
|
extern void reset_transmission_modes(int nestlevel);
|
Add support for asynchronous execution.
This implements asynchronous execution, which runs multiple parts of a
non-parallel-aware Append concurrently rather than serially to improve
performance when possible. Currently, the only node type that can be
run concurrently is a ForeignScan that is an immediate child of such an
Append. In the case where such ForeignScans access data on different
remote servers, this would run those ForeignScans concurrently, and
overlap the remote operations to be performed simultaneously, so it'll
improve the performance especially when the operations involve
time-consuming ones such as remote join and remote aggregation.
We may extend this to other node types such as joins or aggregates over
ForeignScans in the future.
This also adds the support for postgres_fdw, which is enabled by the
table-level/server-level option "async_capable". The default is false.
Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit
is mostly based on the patch proposed by Robert Haas, but also uses
stuff from the patch proposed by Kyotaro Horiguchi and from the patch
proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin
Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and
others.
Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com
Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
2021-03-31 11:45:00 +02:00
|
|
|
extern void process_pending_request(AsyncRequest *areq);
|
2013-03-12 02:31:28 +01:00
|
|
|
|
2013-02-21 11:26:23 +01:00
|
|
|
/* in connection.c */
|
Add support for asynchronous execution.
This implements asynchronous execution, which runs multiple parts of a
non-parallel-aware Append concurrently rather than serially to improve
performance when possible. Currently, the only node type that can be
run concurrently is a ForeignScan that is an immediate child of such an
Append. In the case where such ForeignScans access data on different
remote servers, this would run those ForeignScans concurrently, and
overlap the remote operations to be performed simultaneously, so it'll
improve the performance especially when the operations involve
time-consuming ones such as remote join and remote aggregation.
We may extend this to other node types such as joins or aggregates over
ForeignScans in the future.
This also adds the support for postgres_fdw, which is enabled by the
table-level/server-level option "async_capable". The default is false.
Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit
is mostly based on the patch proposed by Robert Haas, but also uses
stuff from the patch proposed by Kyotaro Horiguchi and from the patch
proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin
Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and
others.
Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com
Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
2021-03-31 11:45:00 +02:00
|
|
|
extern PGconn *GetConnection(UserMapping *user, bool will_prep_stmt,
|
|
|
|
PgFdwConnState **state);
|
2013-02-21 11:26:23 +01:00
|
|
|
extern void ReleaseConnection(PGconn *conn);
|
|
|
|
extern unsigned int GetCursorNumber(PGconn *conn);
|
2013-03-10 19:14:53 +01:00
|
|
|
extern unsigned int GetPrepStmtNumber(PGconn *conn);
|
Allow TRUNCATE command to truncate foreign tables.
This commit introduces new foreign data wrapper API for TRUNCATE.
It extends TRUNCATE command so that it accepts foreign tables as
the targets to truncate and invokes that API. Also it extends postgres_fdw
so that it can issue TRUNCATE command to foreign servers, by adding
new routine for that TRUNCATE API.
The information about options specified in TRUNCATE command, e.g.,
ONLY, CACADE, etc is passed to FDW via API. The list of foreign tables to
truncate is also passed to FDW. FDW truncates the foreign data sources
that the passed foreign tables specify, based on those information.
For example, postgres_fdw constructs TRUNCATE command using them
and issues it to the foreign server.
For performance, TRUNCATE command invokes the FDW routine for
TRUNCATE once per foreign server that foreign tables to truncate belong to.
Author: Kazutaka Onishi, Kohei KaiGai, slightly modified by Fujii Masao
Reviewed-by: Bharath Rupireddy, Michael Paquier, Zhihong Yu, Alvaro Herrera, Stephen Frost, Ashutosh Bapat, Amit Langote, Daniel Gustafsson, Ibrar Ahmed, Fujii Masao
Discussion: https://postgr.es/m/CAOP8fzb_gkReLput7OvOK+8NHgw-RKqNv59vem7=524krQTcWA@mail.gmail.com
Discussion: https://postgr.es/m/CAJuF6cMWDDqU-vn_knZgma+2GMaout68YUgn1uyDnexRhqqM5Q@mail.gmail.com
2021-04-08 13:56:08 +02:00
|
|
|
extern void do_sql_command(PGconn *conn, const char *sql);
|
2024-01-08 20:39:56 +01:00
|
|
|
extern PGresult *pgfdw_get_result(PGconn *conn);
|
Add support for asynchronous execution.
This implements asynchronous execution, which runs multiple parts of a
non-parallel-aware Append concurrently rather than serially to improve
performance when possible. Currently, the only node type that can be
run concurrently is a ForeignScan that is an immediate child of such an
Append. In the case where such ForeignScans access data on different
remote servers, this would run those ForeignScans concurrently, and
overlap the remote operations to be performed simultaneously, so it'll
improve the performance especially when the operations involve
time-consuming ones such as remote join and remote aggregation.
We may extend this to other node types such as joins or aggregates over
ForeignScans in the future.
This also adds the support for postgres_fdw, which is enabled by the
table-level/server-level option "async_capable". The default is false.
Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit
is mostly based on the patch proposed by Robert Haas, but also uses
stuff from the patch proposed by Kyotaro Horiguchi and from the patch
proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin
Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and
others.
Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com
Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com
Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
2021-03-31 11:45:00 +02:00
|
|
|
extern PGresult *pgfdw_exec_query(PGconn *conn, const char *query,
|
|
|
|
PgFdwConnState *state);
|
2014-02-04 03:30:02 +01:00
|
|
|
extern void pgfdw_report_error(int elevel, PGresult *res, PGconn *conn,
|
2019-05-22 19:04:48 +02:00
|
|
|
bool clear, const char *sql);
|
2013-02-21 11:26:23 +01:00
|
|
|
|
|
|
|
/* in option.c */
|
2019-05-22 19:04:48 +02:00
|
|
|
extern int ExtractConnectionOptions(List *defelems,
|
|
|
|
const char **keywords,
|
|
|
|
const char **values);
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
extern List *ExtractExtensionList(const char *extensionsString,
|
2019-05-22 19:04:48 +02:00
|
|
|
bool warnOnMissing);
|
2021-12-24 08:55:11 +01:00
|
|
|
extern char *process_pgfdw_appname(const char *appname);
|
2021-09-07 05:27:30 +02:00
|
|
|
extern char *pgfdw_application_name;
|
2013-02-21 11:26:23 +01:00
|
|
|
|
|
|
|
/* in deparse.c */
|
|
|
|
extern void classifyConditions(PlannerInfo *root,
|
2019-05-22 19:04:48 +02:00
|
|
|
RelOptInfo *baserel,
|
|
|
|
List *input_conds,
|
|
|
|
List **remote_conds,
|
|
|
|
List **local_conds);
|
2013-03-22 00:43:59 +01:00
|
|
|
extern bool is_foreign_expr(PlannerInfo *root,
|
2019-05-22 19:04:48 +02:00
|
|
|
RelOptInfo *baserel,
|
|
|
|
Expr *expr);
|
Avoid postgres_fdw crash for a targetlist entry that's just a Param.
foreign_grouping_ok() is willing to put fairly arbitrary expressions into
the targetlist of a remote SELECT that's doing grouping or aggregation on
the remote side, including expressions that have no foreign component to
them at all. This is possibly a bit dubious from an efficiency standpoint;
but it rises to the level of a crash-causing bug if the expression is just
a Param or non-foreign Var. In that case, the expression will necessarily
also appear in the fdw_exprs list of values we need to send to the remote
server, and then setrefs.c's set_foreignscan_references will mistakenly
replace the fdw_exprs entry with a Var referencing the targetlist result.
The root cause of this problem is bad design in commit e7cb7ee14: it put
logic into set_foreignscan_references that IMV is postgres_fdw-specific,
and yet this bug shows that it isn't postgres_fdw-specific enough. The
transformation being done on fdw_exprs assumes that fdw_exprs is to be
evaluated with the fdw_scan_tlist as input, which is not how postgres_fdw
uses it; yet it could be the right thing for some other FDW. (In the
bigger picture, setrefs.c has no business assuming this for the other
expression fields of a ForeignScan either.)
The right fix therefore would be to expand the FDW API so that the
FDW could inform setrefs.c how it intends to evaluate these various
expressions. We can't change that in the back branches though, and we
also can't just summarily change setrefs.c's behavior there, or we're
likely to break external FDWs.
As a stopgap, therefore, hack up postgres_fdw so that it won't attempt
to send targetlist entries that look exactly like the fdw_exprs entries
they'd produce. In most cases this actually produces a superior plan,
IMO, with less data needing to be transmitted and returned; so we probably
ought to think harder about whether we should ship tlist expressions at
all when they don't contain any foreign Vars or Aggs. But that's an
optimization not a bug fix so I left it for later. One case where this
produces an inferior plan is where the expression in question is actually
a GROUP BY expression: then the restriction prevents us from using remote
grouping. It might be possible to work around that (since that would
reduce to group-by-a-constant on the remote side); but it seems like a
pretty unlikely corner case, so I'm not sure it's worth expending effort
solely to improve that. In any case the right long-term answer is to fix
the API as sketched above, and then revert this hack.
Per bug #15781 from Sean Johnston. Back-patch to v10 where the problem
was introduced.
Discussion: https://postgr.es/m/15781-2601b1002bad087c@postgresql.org
2019-04-27 19:15:54 +02:00
|
|
|
extern bool is_foreign_param(PlannerInfo *root,
|
2019-05-22 19:04:48 +02:00
|
|
|
RelOptInfo *baserel,
|
|
|
|
Expr *expr);
|
Fix postgres_fdw to check shippability of sort clauses properly.
postgres_fdw would push ORDER BY clauses to the remote side without
verifying that the sort operator is safe to ship. Moreover, it failed
to print a suitable USING clause if the sort operator isn't default
for the sort expression's type. The net result of this is that the
remote sort might not have anywhere near the semantics we expect,
which'd be disastrous for locally-performed merge joins in particular.
We addressed similar issues in the context of ORDER BY within an
aggregate function call in commit 7012b132d, but failed to notice
that query-level ORDER BY was broken. Thus, much of the necessary
logic already existed, but it requires refactoring to be usable
in both cases.
Back-patch to all supported branches. In HEAD only, remove the
core code's copy of find_em_expr_for_rel, which is no longer used
and really should never have been pushed into equivclass.c in the
first place.
Ronan Dunklau, per report from David Rowley;
reviews by David Rowley, Ranier Vilela, and myself
Discussion: https://postgr.es/m/CAApHDvr4OeC2DBVY--zVP83-K=bYrTD7F8SZDhN4g+pj2f2S-A@mail.gmail.com
2022-03-31 20:29:24 +02:00
|
|
|
extern bool is_foreign_pathkey(PlannerInfo *root,
|
|
|
|
RelOptInfo *baserel,
|
|
|
|
PathKey *pathkey);
|
Fix interaction of foreign tuple routing with remote triggers.
Without these fixes, changes to the inserted tuple made by remote
triggers are ignored when building local RETURNING tuples.
In the core code, call ExecInitRoutingInfo at a later point from
within ExecInitPartitionInfo so that the FDW callback gets invoked
after the returning list has been built. But move CheckValidResultRel
out of ExecInitRoutingInfo so that it can happen at an earlier stage.
In postgres_fdw, refactor assorted deparsing functions to work with
the RTE rather than the PlannerInfo, which saves us having to
construct a fake PlannerInfo in cases where we don't have a real one.
Then, we can pass down a constructed RTE that yields the correct
deparse result when no real one exists. Unfortunately, this
necessitates a hack that understands how the core code manages RT
indexes for update tuple routing, which is ugly, but we don't have a
better idea right now.
Original report, analysis, and patch by Etsuro Fujita. Heavily
refactored by me. Then worked over some more by Amit Langote.
Discussion: http://postgr.es/m/5AD4882B.10002@lab.ntt.co.jp
2018-05-01 19:21:46 +02:00
|
|
|
extern void deparseInsertSql(StringInfo buf, RangeTblEntry *rte,
|
2019-05-22 19:04:48 +02:00
|
|
|
Index rtindex, Relation rel,
|
|
|
|
List *targetAttrs, bool doNothing,
|
|
|
|
List *withCheckOptionList, List *returningList,
|
2021-01-20 23:05:46 +01:00
|
|
|
List **retrieved_attrs, int *values_end_len);
|
2021-08-05 13:00:00 +02:00
|
|
|
extern void rebuildInsertSql(StringInfo buf, Relation rel,
|
|
|
|
char *orig_query, List *target_attrs,
|
|
|
|
int values_end_len, int num_params,
|
2021-01-20 23:05:46 +01:00
|
|
|
int num_rows);
|
Fix interaction of foreign tuple routing with remote triggers.
Without these fixes, changes to the inserted tuple made by remote
triggers are ignored when building local RETURNING tuples.
In the core code, call ExecInitRoutingInfo at a later point from
within ExecInitPartitionInfo so that the FDW callback gets invoked
after the returning list has been built. But move CheckValidResultRel
out of ExecInitRoutingInfo so that it can happen at an earlier stage.
In postgres_fdw, refactor assorted deparsing functions to work with
the RTE rather than the PlannerInfo, which saves us having to
construct a fake PlannerInfo in cases where we don't have a real one.
Then, we can pass down a constructed RTE that yields the correct
deparse result when no real one exists. Unfortunately, this
necessitates a hack that understands how the core code manages RT
indexes for update tuple routing, which is ugly, but we don't have a
better idea right now.
Original report, analysis, and patch by Etsuro Fujita. Heavily
refactored by me. Then worked over some more by Amit Langote.
Discussion: http://postgr.es/m/5AD4882B.10002@lab.ntt.co.jp
2018-05-01 19:21:46 +02:00
|
|
|
extern void deparseUpdateSql(StringInfo buf, RangeTblEntry *rte,
|
2019-05-22 19:04:48 +02:00
|
|
|
Index rtindex, Relation rel,
|
|
|
|
List *targetAttrs,
|
|
|
|
List *withCheckOptionList, List *returningList,
|
|
|
|
List **retrieved_attrs);
|
2016-03-18 18:48:58 +01:00
|
|
|
extern void deparseDirectUpdateSql(StringInfo buf, PlannerInfo *root,
|
2019-05-22 19:04:48 +02:00
|
|
|
Index rtindex, Relation rel,
|
|
|
|
RelOptInfo *foreignrel,
|
|
|
|
List *targetlist,
|
|
|
|
List *targetAttrs,
|
|
|
|
List *remote_conds,
|
|
|
|
List **params_list,
|
|
|
|
List *returningList,
|
|
|
|
List **retrieved_attrs);
|
Fix interaction of foreign tuple routing with remote triggers.
Without these fixes, changes to the inserted tuple made by remote
triggers are ignored when building local RETURNING tuples.
In the core code, call ExecInitRoutingInfo at a later point from
within ExecInitPartitionInfo so that the FDW callback gets invoked
after the returning list has been built. But move CheckValidResultRel
out of ExecInitRoutingInfo so that it can happen at an earlier stage.
In postgres_fdw, refactor assorted deparsing functions to work with
the RTE rather than the PlannerInfo, which saves us having to
construct a fake PlannerInfo in cases where we don't have a real one.
Then, we can pass down a constructed RTE that yields the correct
deparse result when no real one exists. Unfortunately, this
necessitates a hack that understands how the core code manages RT
indexes for update tuple routing, which is ugly, but we don't have a
better idea right now.
Original report, analysis, and patch by Etsuro Fujita. Heavily
refactored by me. Then worked over some more by Amit Langote.
Discussion: http://postgr.es/m/5AD4882B.10002@lab.ntt.co.jp
2018-05-01 19:21:46 +02:00
|
|
|
extern void deparseDeleteSql(StringInfo buf, RangeTblEntry *rte,
|
2019-05-22 19:04:48 +02:00
|
|
|
Index rtindex, Relation rel,
|
|
|
|
List *returningList,
|
|
|
|
List **retrieved_attrs);
|
2016-03-18 18:48:58 +01:00
|
|
|
extern void deparseDirectDeleteSql(StringInfo buf, PlannerInfo *root,
|
2019-05-22 19:04:48 +02:00
|
|
|
Index rtindex, Relation rel,
|
|
|
|
RelOptInfo *foreignrel,
|
|
|
|
List *remote_conds,
|
|
|
|
List **params_list,
|
|
|
|
List *returningList,
|
|
|
|
List **retrieved_attrs);
|
2013-02-22 16:56:06 +01:00
|
|
|
extern void deparseAnalyzeSizeSql(StringInfo buf, Relation rel);
|
2023-01-07 14:22:09 +01:00
|
|
|
extern void deparseAnalyzeInfoSql(StringInfo buf, Relation rel);
|
2013-03-22 05:31:11 +01:00
|
|
|
extern void deparseAnalyzeSql(StringInfo buf, Relation rel,
|
2022-12-30 23:14:53 +01:00
|
|
|
PgFdwSamplingMethod sample_method,
|
|
|
|
double sample_frac,
|
2019-05-22 19:04:48 +02:00
|
|
|
List **retrieved_attrs);
|
Allow TRUNCATE command to truncate foreign tables.
This commit introduces new foreign data wrapper API for TRUNCATE.
It extends TRUNCATE command so that it accepts foreign tables as
the targets to truncate and invokes that API. Also it extends postgres_fdw
so that it can issue TRUNCATE command to foreign servers, by adding
new routine for that TRUNCATE API.
The information about options specified in TRUNCATE command, e.g.,
ONLY, CACADE, etc is passed to FDW via API. The list of foreign tables to
truncate is also passed to FDW. FDW truncates the foreign data sources
that the passed foreign tables specify, based on those information.
For example, postgres_fdw constructs TRUNCATE command using them
and issues it to the foreign server.
For performance, TRUNCATE command invokes the FDW routine for
TRUNCATE once per foreign server that foreign tables to truncate belong to.
Author: Kazutaka Onishi, Kohei KaiGai, slightly modified by Fujii Masao
Reviewed-by: Bharath Rupireddy, Michael Paquier, Zhihong Yu, Alvaro Herrera, Stephen Frost, Ashutosh Bapat, Amit Langote, Daniel Gustafsson, Ibrar Ahmed, Fujii Masao
Discussion: https://postgr.es/m/CAOP8fzb_gkReLput7OvOK+8NHgw-RKqNv59vem7=524krQTcWA@mail.gmail.com
Discussion: https://postgr.es/m/CAJuF6cMWDDqU-vn_knZgma+2GMaout68YUgn1uyDnexRhqqM5Q@mail.gmail.com
2021-04-08 13:56:08 +02:00
|
|
|
extern void deparseTruncateSql(StringInfo buf,
|
|
|
|
List *rels,
|
|
|
|
DropBehavior behavior,
|
|
|
|
bool restart_seqs);
|
2014-07-10 21:01:31 +02:00
|
|
|
extern void deparseStringLiteral(StringInfo buf, const char *val);
|
Fix postgres_fdw to check shippability of sort clauses properly.
postgres_fdw would push ORDER BY clauses to the remote side without
verifying that the sort operator is safe to ship. Moreover, it failed
to print a suitable USING clause if the sort operator isn't default
for the sort expression's type. The net result of this is that the
remote sort might not have anywhere near the semantics we expect,
which'd be disastrous for locally-performed merge joins in particular.
We addressed similar issues in the context of ORDER BY within an
aggregate function call in commit 7012b132d, but failed to notice
that query-level ORDER BY was broken. Thus, much of the necessary
logic already existed, but it requires refactoring to be usable
in both cases.
Back-patch to all supported branches. In HEAD only, remove the
core code's copy of find_em_expr_for_rel, which is no longer used
and really should never have been pushed into equivclass.c in the
first place.
Ronan Dunklau, per report from David Rowley;
reviews by David Rowley, Ranier Vilela, and myself
Discussion: https://postgr.es/m/CAApHDvr4OeC2DBVY--zVP83-K=bYrTD7F8SZDhN4g+pj2f2S-A@mail.gmail.com
2022-03-31 20:29:24 +02:00
|
|
|
extern EquivalenceMember *find_em_for_rel(PlannerInfo *root,
|
|
|
|
EquivalenceClass *ec,
|
|
|
|
RelOptInfo *rel);
|
|
|
|
extern EquivalenceMember *find_em_for_rel_target(PlannerInfo *root,
|
|
|
|
EquivalenceClass *ec,
|
|
|
|
RelOptInfo *rel);
|
2016-10-21 15:54:29 +02:00
|
|
|
extern List *build_tlist_to_deparse(RelOptInfo *foreignrel);
|
2016-01-30 16:32:38 +01:00
|
|
|
extern void deparseSelectStmtForRel(StringInfo buf, PlannerInfo *root,
|
2022-09-22 22:59:20 +02:00
|
|
|
RelOptInfo *rel, List *tlist,
|
2019-05-22 19:04:48 +02:00
|
|
|
List *remote_conds, List *pathkeys,
|
|
|
|
bool has_final_sort, bool has_limit,
|
|
|
|
bool is_subquery,
|
|
|
|
List **retrieved_attrs, List **params_list);
|
2017-06-22 18:44:53 +02:00
|
|
|
extern const char *get_jointype_name(JoinType jointype);
|
Allow postgres_fdw to ship extension funcs/operators for remote execution.
The user can whitelist specified extension(s) in the foreign server's
options, whereupon we will treat immutable functions and operators of those
extensions as candidates to be sent for remote execution.
Whitelisting an extension in this way basically promises that the extension
exists on the remote server and behaves compatibly with the local instance.
We have no way to prove that formally, so we have to rely on the user to
get it right. But this seems like something that people can usually get
right in practice.
We might in future allow functions and operators to be whitelisted
individually, but extension granularity is a very convenient special case,
so it got done first.
The patch as-committed lacks any regression tests, which is unfortunate,
but introducing dependencies on other extensions for testing purposes
would break "make installcheck" scenarios, which is worse. I have some
ideas about klugy ways around that, but it seems like material for a
separate patch. For the moment, leave the problem open.
Paul Ramsey, hacked up a bit more by me
2015-11-04 00:42:02 +01:00
|
|
|
|
|
|
|
/* in shippable.c */
|
|
|
|
extern bool is_builtin(Oid objectId);
|
|
|
|
extern bool is_shippable(Oid objectId, Oid classId, PgFdwRelationInfo *fpinfo);
|
2013-02-21 11:26:23 +01:00
|
|
|
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
#endif /* POSTGRES_FDW_H */
|