2010-05-12 04:19:11 +02:00
|
|
|
/*
|
|
|
|
* function.c
|
|
|
|
*
|
|
|
|
* server-side function support
|
2010-07-03 16:23:14 +02:00
|
|
|
*
|
2024-01-04 02:49:05 +01:00
|
|
|
* Copyright (c) 2010-2024, PostgreSQL Global Development Group
|
2015-03-11 03:33:25 +01:00
|
|
|
* src/bin/pg_upgrade/function.c
|
2010-05-12 04:19:11 +02:00
|
|
|
*/
|
|
|
|
|
Create libpgcommon, and move pg_malloc et al to it
libpgcommon is a new static library to allow sharing code among the
various frontend programs and backend; this lets us eliminate duplicate
implementations of common routines. We avoid libpgport, because that's
intended as a place for porting issues; per discussion, it seems better
to keep them separate.
The first use case, and the only implemented by this patch, is pg_malloc
and friends, which many frontend programs were already using.
At the same time, we can use this to provide palloc emulation functions
for the frontend; this way, some palloc-using files in the backend can
also be used by the frontend cleanly. To do this, we change palloc() in
the backend to be a function instead of a macro on top of
MemoryContextAlloc(). This was previously believed to cause loss of
performance, but this implementation has been tweaked by Tom and Andres
so that on modern compilers it provides a slight improvement over the
previous one.
This lets us clean up some places that were already with
localized hacks.
Most of the pg_malloc/palloc changes in this patch were authored by
Andres Freund. Zoltán Böszörményi also independently provided a form of
that. libpgcommon infrastructure was authored by Álvaro.
2013-02-12 14:33:40 +01:00
|
|
|
#include "postgres_fe.h"
|
2011-08-27 03:16:24 +02:00
|
|
|
|
2010-05-12 04:19:11 +02:00
|
|
|
#include "access/transam.h"
|
2018-04-08 19:59:52 +02:00
|
|
|
#include "catalog/pg_language_d.h"
|
2024-02-16 21:05:36 +01:00
|
|
|
#include "common/int.h"
|
2019-10-23 06:08:53 +02:00
|
|
|
#include "pg_upgrade.h"
|
Enforce a specific order for probing library loadability in pg_upgrade.
pg_upgrade checks whether all the shared libraries used in the old cluster
are also available in the new one by issuing LOAD for each library name.
Previously, it cared not what order it did the LOADs in. Ideally it
should not have to care, but currently the transform modules in contrib
fail unless both the language and datatype modules they depend on are
loaded first. A backend-side solution for that looks possible but
probably not back-patchable, so as a stopgap measure, let's do the LOAD
tests in order by library name length. That should fix the problem for
reasonably-named transform modules, eg "hstore_plpython" will be loaded
after both "hstore" and "plpython". (Yeah, it's a hack.)
In a larger sense, having a predictable order of these probes is a good
thing, since it will make upgrades predictably work or not work in the
face of inter-library dependencies. Also, this patch replaces O(N^2)
de-duplication logic with O(N log N) logic, which could matter in
installations with very many databases. So I don't foresee reverting this
even after we have a proper fix for the library-dependency problem.
In passing, improve a couple of SQL queries used here.
Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib
modules failed. Back-patch to 9.5 where transform modules were introduced.
Discussion: <f7ac29f3-515c-2a44-21c5-ec925053265f@dunslane.net>
2016-10-03 16:07:39 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* qsort comparator for pointers to library names
|
|
|
|
*
|
2018-07-28 18:33:54 +02:00
|
|
|
* We sort first by name length, then alphabetically for names of the
|
|
|
|
* same length, then database array index. This is to ensure that, eg,
|
|
|
|
* "hstore_plpython" sorts after both "hstore" and "plpython"; otherwise
|
|
|
|
* transform modules will probably fail their LOAD tests. (The backend
|
|
|
|
* ought to cope with that consideration, but it doesn't yet, and even
|
|
|
|
* when it does it'll still be a good idea to have a predictable order of
|
|
|
|
* probing here.)
|
Enforce a specific order for probing library loadability in pg_upgrade.
pg_upgrade checks whether all the shared libraries used in the old cluster
are also available in the new one by issuing LOAD for each library name.
Previously, it cared not what order it did the LOADs in. Ideally it
should not have to care, but currently the transform modules in contrib
fail unless both the language and datatype modules they depend on are
loaded first. A backend-side solution for that looks possible but
probably not back-patchable, so as a stopgap measure, let's do the LOAD
tests in order by library name length. That should fix the problem for
reasonably-named transform modules, eg "hstore_plpython" will be loaded
after both "hstore" and "plpython". (Yeah, it's a hack.)
In a larger sense, having a predictable order of these probes is a good
thing, since it will make upgrades predictably work or not work in the
face of inter-library dependencies. Also, this patch replaces O(N^2)
de-duplication logic with O(N log N) logic, which could matter in
installations with very many databases. So I don't foresee reverting this
even after we have a proper fix for the library-dependency problem.
In passing, improve a couple of SQL queries used here.
Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib
modules failed. Back-patch to 9.5 where transform modules were introduced.
Discussion: <f7ac29f3-515c-2a44-21c5-ec925053265f@dunslane.net>
2016-10-03 16:07:39 +02:00
|
|
|
*/
|
|
|
|
static int
|
|
|
|
library_name_compare(const void *p1, const void *p2)
|
|
|
|
{
|
2018-07-28 18:33:54 +02:00
|
|
|
const char *str1 = ((const LibraryInfo *) p1)->name;
|
|
|
|
const char *str2 = ((const LibraryInfo *) p2)->name;
|
2024-02-16 21:05:36 +01:00
|
|
|
size_t slen1 = strlen(str1);
|
|
|
|
size_t slen2 = strlen(str2);
|
2018-07-28 18:33:54 +02:00
|
|
|
int cmp = strcmp(str1, str2);
|
2019-01-24 21:58:37 +01:00
|
|
|
|
Enforce a specific order for probing library loadability in pg_upgrade.
pg_upgrade checks whether all the shared libraries used in the old cluster
are also available in the new one by issuing LOAD for each library name.
Previously, it cared not what order it did the LOADs in. Ideally it
should not have to care, but currently the transform modules in contrib
fail unless both the language and datatype modules they depend on are
loaded first. A backend-side solution for that looks possible but
probably not back-patchable, so as a stopgap measure, let's do the LOAD
tests in order by library name length. That should fix the problem for
reasonably-named transform modules, eg "hstore_plpython" will be loaded
after both "hstore" and "plpython". (Yeah, it's a hack.)
In a larger sense, having a predictable order of these probes is a good
thing, since it will make upgrades predictably work or not work in the
face of inter-library dependencies. Also, this patch replaces O(N^2)
de-duplication logic with O(N log N) logic, which could matter in
installations with very many databases. So I don't foresee reverting this
even after we have a proper fix for the library-dependency problem.
In passing, improve a couple of SQL queries used here.
Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib
modules failed. Back-patch to 9.5 where transform modules were introduced.
Discussion: <f7ac29f3-515c-2a44-21c5-ec925053265f@dunslane.net>
2016-10-03 16:07:39 +02:00
|
|
|
if (slen1 != slen2)
|
2024-02-16 21:05:36 +01:00
|
|
|
return pg_cmp_size(slen1, slen2);
|
2018-07-28 18:33:54 +02:00
|
|
|
if (cmp != 0)
|
|
|
|
return cmp;
|
2024-02-16 21:05:36 +01:00
|
|
|
return pg_cmp_s32(((const LibraryInfo *) p1)->dbnum,
|
|
|
|
((const LibraryInfo *) p2)->dbnum);
|
Enforce a specific order for probing library loadability in pg_upgrade.
pg_upgrade checks whether all the shared libraries used in the old cluster
are also available in the new one by issuing LOAD for each library name.
Previously, it cared not what order it did the LOADs in. Ideally it
should not have to care, but currently the transform modules in contrib
fail unless both the language and datatype modules they depend on are
loaded first. A backend-side solution for that looks possible but
probably not back-patchable, so as a stopgap measure, let's do the LOAD
tests in order by library name length. That should fix the problem for
reasonably-named transform modules, eg "hstore_plpython" will be loaded
after both "hstore" and "plpython". (Yeah, it's a hack.)
In a larger sense, having a predictable order of these probes is a good
thing, since it will make upgrades predictably work or not work in the
face of inter-library dependencies. Also, this patch replaces O(N^2)
de-duplication logic with O(N log N) logic, which could matter in
installations with very many databases. So I don't foresee reverting this
even after we have a proper fix for the library-dependency problem.
In passing, improve a couple of SQL queries used here.
Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib
modules failed. Back-patch to 9.5 where transform modules were introduced.
Discussion: <f7ac29f3-515c-2a44-21c5-ec925053265f@dunslane.net>
2016-10-03 16:07:39 +02:00
|
|
|
}
|
2010-05-12 04:19:11 +02:00
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
* get_loadable_libraries()
|
|
|
|
*
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2023-10-26 03:24:16 +02:00
|
|
|
* Fetch the names of all old libraries containing either C-language functions
|
|
|
|
* or are corresponding to logical replication output plugins.
|
|
|
|
*
|
2010-05-12 04:19:11 +02:00
|
|
|
* We will later check that they all exist in the new installation.
|
|
|
|
*/
|
|
|
|
void
|
2010-10-19 23:38:16 +02:00
|
|
|
get_loadable_libraries(void)
|
2010-05-12 04:19:11 +02:00
|
|
|
{
|
|
|
|
PGresult **ress;
|
|
|
|
int totaltups;
|
|
|
|
int dbnum;
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2023-10-26 03:24:16 +02:00
|
|
|
int n_libinfos;
|
2012-06-10 21:20:04 +02:00
|
|
|
|
2011-11-01 18:49:03 +01:00
|
|
|
ress = (PGresult **) pg_malloc(old_cluster.dbarr.ndbs * sizeof(PGresult *));
|
2010-05-12 04:19:11 +02:00
|
|
|
totaltups = 0;
|
|
|
|
|
|
|
|
/* Fetch all library names, removing duplicates within each DB */
|
2011-01-01 18:06:36 +01:00
|
|
|
for (dbnum = 0; dbnum < old_cluster.dbarr.ndbs; dbnum++)
|
2010-05-12 04:19:11 +02:00
|
|
|
{
|
2011-01-01 18:06:36 +01:00
|
|
|
DbInfo *active_db = &old_cluster.dbarr.dbs[dbnum];
|
|
|
|
PGconn *conn = connectToServer(&old_cluster, active_db->db_name);
|
2010-05-12 04:19:11 +02:00
|
|
|
|
2012-05-27 12:35:10 +02:00
|
|
|
/*
|
Enforce a specific order for probing library loadability in pg_upgrade.
pg_upgrade checks whether all the shared libraries used in the old cluster
are also available in the new one by issuing LOAD for each library name.
Previously, it cared not what order it did the LOADs in. Ideally it
should not have to care, but currently the transform modules in contrib
fail unless both the language and datatype modules they depend on are
loaded first. A backend-side solution for that looks possible but
probably not back-patchable, so as a stopgap measure, let's do the LOAD
tests in order by library name length. That should fix the problem for
reasonably-named transform modules, eg "hstore_plpython" will be loaded
after both "hstore" and "plpython". (Yeah, it's a hack.)
In a larger sense, having a predictable order of these probes is a good
thing, since it will make upgrades predictably work or not work in the
face of inter-library dependencies. Also, this patch replaces O(N^2)
de-duplication logic with O(N log N) logic, which could matter in
installations with very many databases. So I don't foresee reverting this
even after we have a proper fix for the library-dependency problem.
In passing, improve a couple of SQL queries used here.
Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib
modules failed. Back-patch to 9.5 where transform modules were introduced.
Discussion: <f7ac29f3-515c-2a44-21c5-ec925053265f@dunslane.net>
2016-10-03 16:07:39 +02:00
|
|
|
* Fetch all libraries containing non-built-in C functions in this DB.
|
2012-05-27 12:35:10 +02:00
|
|
|
*/
|
2010-10-19 23:38:16 +02:00
|
|
|
ress[dbnum] = executeQueryOrDie(conn,
|
2010-05-12 04:19:11 +02:00
|
|
|
"SELECT DISTINCT probin "
|
Enforce a specific order for probing library loadability in pg_upgrade.
pg_upgrade checks whether all the shared libraries used in the old cluster
are also available in the new one by issuing LOAD for each library name.
Previously, it cared not what order it did the LOADs in. Ideally it
should not have to care, but currently the transform modules in contrib
fail unless both the language and datatype modules they depend on are
loaded first. A backend-side solution for that looks possible but
probably not back-patchable, so as a stopgap measure, let's do the LOAD
tests in order by library name length. That should fix the problem for
reasonably-named transform modules, eg "hstore_plpython" will be loaded
after both "hstore" and "plpython". (Yeah, it's a hack.)
In a larger sense, having a predictable order of these probes is a good
thing, since it will make upgrades predictably work or not work in the
face of inter-library dependencies. Also, this patch replaces O(N^2)
de-duplication logic with O(N log N) logic, which could matter in
installations with very many databases. So I don't foresee reverting this
even after we have a proper fix for the library-dependency problem.
In passing, improve a couple of SQL queries used here.
Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib
modules failed. Back-patch to 9.5 where transform modules were introduced.
Discussion: <f7ac29f3-515c-2a44-21c5-ec925053265f@dunslane.net>
2016-10-03 16:07:39 +02:00
|
|
|
"FROM pg_catalog.pg_proc "
|
|
|
|
"WHERE prolang = %u AND "
|
2011-02-10 01:17:33 +01:00
|
|
|
"probin IS NOT NULL AND "
|
|
|
|
"oid >= %u;",
|
Enforce a specific order for probing library loadability in pg_upgrade.
pg_upgrade checks whether all the shared libraries used in the old cluster
are also available in the new one by issuing LOAD for each library name.
Previously, it cared not what order it did the LOADs in. Ideally it
should not have to care, but currently the transform modules in contrib
fail unless both the language and datatype modules they depend on are
loaded first. A backend-side solution for that looks possible but
probably not back-patchable, so as a stopgap measure, let's do the LOAD
tests in order by library name length. That should fix the problem for
reasonably-named transform modules, eg "hstore_plpython" will be loaded
after both "hstore" and "plpython". (Yeah, it's a hack.)
In a larger sense, having a predictable order of these probes is a good
thing, since it will make upgrades predictably work or not work in the
face of inter-library dependencies. Also, this patch replaces O(N^2)
de-duplication logic with O(N log N) logic, which could matter in
installations with very many databases. So I don't foresee reverting this
even after we have a proper fix for the library-dependency problem.
In passing, improve a couple of SQL queries used here.
Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib
modules failed. Back-patch to 9.5 where transform modules were introduced.
Discussion: <f7ac29f3-515c-2a44-21c5-ec925053265f@dunslane.net>
2016-10-03 16:07:39 +02:00
|
|
|
ClanguageId,
|
2010-05-12 04:19:11 +02:00
|
|
|
FirstNormalObjectId);
|
|
|
|
totaltups += PQntuples(ress[dbnum]);
|
|
|
|
|
|
|
|
PQfinish(conn);
|
|
|
|
}
|
|
|
|
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2023-10-26 03:24:16 +02:00
|
|
|
/*
|
|
|
|
* Allocate memory for required libraries and logical replication output
|
|
|
|
* plugins.
|
|
|
|
*/
|
|
|
|
n_libinfos = totaltups + count_old_cluster_logical_slots();
|
|
|
|
os_info.libraries = (LibraryInfo *) pg_malloc(sizeof(LibraryInfo) * n_libinfos);
|
2010-05-12 04:19:11 +02:00
|
|
|
totaltups = 0;
|
|
|
|
|
2011-01-01 18:06:36 +01:00
|
|
|
for (dbnum = 0; dbnum < old_cluster.dbarr.ndbs; dbnum++)
|
2010-05-12 04:19:11 +02:00
|
|
|
{
|
|
|
|
PGresult *res = ress[dbnum];
|
|
|
|
int ntups;
|
|
|
|
int rowno;
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2023-10-26 03:24:16 +02:00
|
|
|
LogicalSlotInfoArr *slot_arr = &old_cluster.dbarr.dbs[dbnum].slot_arr;
|
2010-05-12 04:19:11 +02:00
|
|
|
|
|
|
|
ntups = PQntuples(res);
|
|
|
|
for (rowno = 0; rowno < ntups; rowno++)
|
|
|
|
{
|
|
|
|
char *lib = PQgetvalue(res, rowno, 0);
|
|
|
|
|
2018-07-28 18:33:54 +02:00
|
|
|
os_info.libraries[totaltups].name = pg_strdup(lib);
|
|
|
|
os_info.libraries[totaltups].dbnum = dbnum;
|
|
|
|
|
|
|
|
totaltups++;
|
2010-05-12 04:19:11 +02:00
|
|
|
}
|
|
|
|
PQclear(res);
|
Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.
If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.
The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.
Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2023-10-26 03:24:16 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Store the names of output plugins as well. There is a possibility
|
|
|
|
* that duplicated plugins are set, but the consumer function
|
|
|
|
* check_loadable_libraries() will avoid checking the same library, so
|
|
|
|
* we do not have to consider their uniqueness here.
|
|
|
|
*/
|
|
|
|
for (int slotno = 0; slotno < slot_arr->nslots; slotno++)
|
|
|
|
{
|
|
|
|
if (slot_arr->slots[slotno].invalid)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
os_info.libraries[totaltups].name = pg_strdup(slot_arr->slots[slotno].plugin);
|
|
|
|
os_info.libraries[totaltups].dbnum = dbnum;
|
|
|
|
|
|
|
|
totaltups++;
|
|
|
|
}
|
2010-05-12 04:19:11 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
pg_free(ress);
|
Enforce a specific order for probing library loadability in pg_upgrade.
pg_upgrade checks whether all the shared libraries used in the old cluster
are also available in the new one by issuing LOAD for each library name.
Previously, it cared not what order it did the LOADs in. Ideally it
should not have to care, but currently the transform modules in contrib
fail unless both the language and datatype modules they depend on are
loaded first. A backend-side solution for that looks possible but
probably not back-patchable, so as a stopgap measure, let's do the LOAD
tests in order by library name length. That should fix the problem for
reasonably-named transform modules, eg "hstore_plpython" will be loaded
after both "hstore" and "plpython". (Yeah, it's a hack.)
In a larger sense, having a predictable order of these probes is a good
thing, since it will make upgrades predictably work or not work in the
face of inter-library dependencies. Also, this patch replaces O(N^2)
de-duplication logic with O(N log N) logic, which could matter in
installations with very many databases. So I don't foresee reverting this
even after we have a proper fix for the library-dependency problem.
In passing, improve a couple of SQL queries used here.
Per complaint from Andrew Dunstan that pg_upgrade'ing the transform contrib
modules failed. Back-patch to 9.5 where transform modules were introduced.
Discussion: <f7ac29f3-515c-2a44-21c5-ec925053265f@dunslane.net>
2016-10-03 16:07:39 +02:00
|
|
|
|
|
|
|
os_info.num_libraries = totaltups;
|
2010-05-12 04:19:11 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
* check_loadable_libraries()
|
|
|
|
*
|
|
|
|
* Check that the new cluster contains all required libraries.
|
|
|
|
* We do this by actually trying to LOAD each one, thereby testing
|
|
|
|
* compatibility as well as presence.
|
|
|
|
*/
|
|
|
|
void
|
2010-10-19 23:38:16 +02:00
|
|
|
check_loadable_libraries(void)
|
2010-05-12 04:19:11 +02:00
|
|
|
{
|
2011-01-01 18:06:36 +01:00
|
|
|
PGconn *conn = connectToServer(&new_cluster, "template1");
|
2010-05-12 04:19:11 +02:00
|
|
|
int libnum;
|
2018-07-28 18:33:54 +02:00
|
|
|
int was_load_failure = false;
|
2010-05-12 04:19:11 +02:00
|
|
|
FILE *script = NULL;
|
|
|
|
char output_path[MAXPGPATH];
|
|
|
|
|
2010-10-19 23:38:16 +02:00
|
|
|
prep_status("Checking for presence of required libraries");
|
2010-05-12 04:19:11 +02:00
|
|
|
|
pg_upgrade: Move all the files generated internally to a subdirectory
Historically, the location of any files generated by pg_upgrade, as of
the per-database logs and internal dumps, has been the current working
directory, leaving all those files behind when using --retain or on a
failure.
Putting all those contents in a targeted subdirectory makes the whole
easier to debug, and simplifies the code in charge of cleaning up the
logs. Note that another reason is that this facilitates the move of
pg_upgrade to TAP with a fixed location for all the logs to grab if the
test fails repeatedly.
Initially, we thought about being able to specify the output directory
with a new option, but we have settled on using a subdirectory located
at the root of the new cluster's data folder, "pg_upgrade_output.d",
instead, as at the end the new data directory is the location of all the
data generated by pg_upgrade. There is a take with group permissions
here though: if the new data folder has been initialized with this
option, we need to create all the files and paths with the correct
permissions or a base backup taken after a pg_upgrade --retain would
fail, meaning that GetDataDirectoryCreatePerm() has to be called before
creating the log paths, before a couple of sanity checks on the clusters
and before getting the socket directory for the cluster's host settings.
The idea of the new location is based on a suggestion from Peter
Eisentraut.
Also thanks to Andrew Dunstan, Peter Eisentraut, Daniel Gustafsson, Tom
Lane and Bruce Momjian for the discussion (in alphabetical order).
Author: Justin Pryzby
Discussion: https://postgr.es/m/20211212025017.GN17618@telsasoft.com
2022-02-06 04:27:29 +01:00
|
|
|
snprintf(output_path, sizeof(output_path), "%s/%s",
|
|
|
|
log_opts.basedir, "loadable_libraries.txt");
|
2010-05-12 04:19:11 +02:00
|
|
|
|
2018-07-28 18:33:54 +02:00
|
|
|
/*
|
|
|
|
* Now we want to sort the library names into order. This avoids multiple
|
|
|
|
* probes of the same library, and ensures that libraries are probed in a
|
|
|
|
* consistent order, which is important for reproducible behavior if one
|
|
|
|
* library depends on another.
|
|
|
|
*/
|
2023-02-07 06:53:05 +01:00
|
|
|
qsort(os_info.libraries, os_info.num_libraries,
|
2018-07-28 18:33:54 +02:00
|
|
|
sizeof(LibraryInfo), library_name_compare);
|
|
|
|
|
2010-10-19 23:38:16 +02:00
|
|
|
for (libnum = 0; libnum < os_info.num_libraries; libnum++)
|
2010-05-12 04:19:11 +02:00
|
|
|
{
|
2018-07-28 18:33:54 +02:00
|
|
|
char *lib = os_info.libraries[libnum].name;
|
2010-05-12 04:19:11 +02:00
|
|
|
int llen = strlen(lib);
|
2012-01-25 15:35:17 +01:00
|
|
|
char cmd[7 + 2 * MAXPGPATH + 1];
|
2010-05-12 04:19:11 +02:00
|
|
|
PGresult *res;
|
|
|
|
|
2018-07-28 18:33:54 +02:00
|
|
|
/* Did the library name change? Probe it. */
|
|
|
|
if (libnum == 0 || strcmp(lib, os_info.libraries[libnum - 1].name) != 0)
|
2010-05-12 04:19:11 +02:00
|
|
|
{
|
2018-07-28 18:33:54 +02:00
|
|
|
strcpy(cmd, "LOAD '");
|
|
|
|
PQescapeStringConn(conn, cmd + strlen(cmd), lib, llen, NULL);
|
|
|
|
strcat(cmd, "'");
|
2019-01-24 21:58:37 +01:00
|
|
|
|
2018-07-28 18:33:54 +02:00
|
|
|
res = PQexec(conn, cmd);
|
2019-01-24 21:58:37 +01:00
|
|
|
|
2018-07-28 18:33:54 +02:00
|
|
|
if (PQresultStatus(res) != PGRES_COMMAND_OK)
|
|
|
|
{
|
|
|
|
was_load_failure = true;
|
2019-01-24 21:58:37 +01:00
|
|
|
|
2018-07-28 18:33:54 +02:00
|
|
|
if (script == NULL && (script = fopen_priv(output_path, "w")) == NULL)
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
2022-07-12 21:17:44 +02:00
|
|
|
pg_fatal("could not open file \"%s\": %s",
|
2018-07-28 18:33:54 +02:00
|
|
|
output_path, strerror(errno));
|
|
|
|
fprintf(script, _("could not load library \"%s\": %s"),
|
|
|
|
lib,
|
|
|
|
PQerrorMessage(conn));
|
|
|
|
}
|
|
|
|
else
|
|
|
|
was_load_failure = false;
|
2019-01-24 21:58:37 +01:00
|
|
|
|
2018-07-28 18:33:54 +02:00
|
|
|
PQclear(res);
|
2010-05-12 04:19:11 +02:00
|
|
|
}
|
|
|
|
|
2018-07-28 18:33:54 +02:00
|
|
|
if (was_load_failure)
|
2019-10-09 04:16:48 +02:00
|
|
|
fprintf(script, _("In database: %s\n"),
|
2018-07-28 18:33:54 +02:00
|
|
|
old_cluster.dbarr.dbs[os_info.libraries[libnum].dbnum].db_name);
|
2010-05-12 04:19:11 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
PQfinish(conn);
|
|
|
|
|
2022-08-31 13:06:50 +02:00
|
|
|
if (script)
|
2010-05-12 04:19:11 +02:00
|
|
|
{
|
|
|
|
fclose(script);
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
2022-07-12 21:17:44 +02:00
|
|
|
pg_log(PG_REPORT, "fatal");
|
2013-10-02 03:24:56 +02:00
|
|
|
pg_fatal("Your installation references loadable libraries that are missing from the\n"
|
2011-07-12 06:13:51 +02:00
|
|
|
"new installation. You can add these libraries to the new installation,\n"
|
|
|
|
"or remove the functions using them from the old installation. A list of\n"
|
|
|
|
"problem libraries is in the file:\n"
|
Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally. As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.
Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c. Adjust the logging functions
to supply a newline at the right places.
(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)
There are some consequent minor changes in output. The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted. It didn't
seem worth being bug-compatible with that.
Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut
Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
2022-07-12 21:17:44 +02:00
|
|
|
" %s", output_path);
|
2010-05-12 04:19:11 +02:00
|
|
|
}
|
|
|
|
else
|
2010-10-19 23:38:16 +02:00
|
|
|
check_ok();
|
2010-05-12 04:19:11 +02:00
|
|
|
}
|