rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/bin/pg_dump/common.c

996 lines
25 KiB
C
Raw Normal View History

Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* common.c
Clean up after recent pg_dump patches. Fix entirely broken handling of va_list printing routines, update some out-of-date comments, fix some bogus inclusion orders, fix NLS declarations, fix missed realloc calls.
2011-11-30 02:41:06 +01:00
* Catalog routines used by pg_dump; long ago these were shared
Add --section option to pg_dump and pg_restore. Valid values are --pre-data, data and post-data. The option can be given more than once. --schema-only is equivalent to --section=pre-data --section=post-data. --data-only is equivalent to --section=data. Andrew Dunstan, reviewed by Joachim Wieland and Josh Berkus.
2011-12-17 01:09:38 +01:00
* by another dump tool, but not anymore.
First pass at schema-fying pg_dump/pg_restore. Much to do still, but the basic capability seems to work.
2002-05-11 00:36:27 +02:00
*
Update copyright for 2016 Backpatch certain files through 9.1
2016-01-02 19:33:40 +01:00
* Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*
* IDENTIFICATION
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/bin/pg_dump/common.c
- Support for relkind = RELKIND_VIEW. - Use symbols for tests on relkind (ie. use RELKIND_VIEW, not 'v') - Fix bug in support for -b option (== --blobs). - Dump views as views (using 'create view'). - Remove 'isViewRule' since we check the relkind when getting tables. - Now uses temp table 'pgdump_oid' rather than 'pg_dump_oid' (errors otherwise). - Added extra param for specifying handling of OID=0 and which typename to output. - Fixed bug in SQL scanner when SQL contained braces. (in rules) - Use format_type function wherever possible
2000-09-15 06:35:16 +02:00
*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*-------------------------------------------------------------------------
*/
pg_dump: Reduce use of global variables Most pg_dump.c global variables, which were passed down individually to dumping routines, are now grouped as members of the new DumpOptions struct, which is used as a local variable and passed down into routines that need it. This helps future development efforts; in particular it is said to enable a mode in which a parallel pg_dump run can output multiple streams, and have them restored in parallel. Also take the opportunity to clean up the pg_dump header files somewhat, to avoid circularity. Author: Joachim Wieland, revised by Álvaro Herrera Reviewed by Peter Eisentraut
2014-10-14 20:00:55 +02:00
#include "postgres_fe.h"
Clean up after recent pg_dump patches. Fix entirely broken handling of va_list printing routines, update some out-of-date comments, fix some bogus inclusion orders, fix NLS declarations, fix missed realloc calls.
2011-11-30 02:41:06 +01:00
#include "pg_backup_archiver.h"
Move some pg_dump function around. Move functions used only by pg_dump and pg_restore from dumputils.c to a new file, pg_backup_utils.c. dumputils.c is linked into psql and some programs in bin/scripts, so it seems good to keep it slim. The parallel functionality is moved to parallel.c, as is exit_horribly, because the interesting code in exit_horribly is parallel-related. This refactoring gets rid of the on_exit_msg_func function pointer. It was problematic, because a modern gcc version with -Wmissing-format-attribute complained if it wasn't marked with PF_PRINTF_ATTRIBUTE, but the ancient gcc version that Tom Lane's old HP-UX box has didn't accept that attribute on a function pointer, and gave an error. We still use a similar function pointer trick for getLocalPQBuffer() function, to use a thread-local version of that in parallel mode on Windows, but that dodges the problem because it doesn't take printf-like arguments.
2013-03-27 17:10:40 +01:00
#include "pg_backup_utils.h"
pg_dump: Reduce use of global variables Most pg_dump.c global variables, which were passed down individually to dumping routines, are now grouped as members of the new DumpOptions struct, which is used as a local variable and passed down into routines that need it. This helps future development efforts; in particular it is said to enable a mode in which a parallel pg_dump run can output multiple streams, and have them restored in parallel. Also take the opportunity to clean up the pg_dump header files somewhat, to avoid circularity. Author: Joachim Wieland, revised by Álvaro Herrera Reviewed by Peter Eisentraut
2014-10-14 20:00:55 +02:00
#include "pg_dump.h"
Work around naming conflict between zlib and OpenSSL by tweaking inclusion order. Remove some unnecessary #includes (that duplicate c.h).
2003-12-08 17:39:05 +01:00
Reduce the need for frontend programs to include "postgres.h" by refactoring inclusions in src/include/catalog/*.h files. The main idea here is to push function declarations for src/backend/catalog/*.c files into separate headers, rather than sticking them into the corresponding catalog definition file as has been done in the past. This commit only carries out that idea fully for pg_proc, pg_type and pg_conversion, but that's enough for the moment --- if pg_list.h ever becomes unsafe for frontend code to include, we'll need to work a bit more. Zdenek Kotala
2008-03-27 04:57:34 +01:00
#include <ctype.h>
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
#include "catalog/pg_class.h"
Create src/fe_utils/, and move stuff into there from pg_dump's dumputils. Per discussion, we want to create a static library and put the stuff into it that until now has been shared across src/bin/ directories by ad-hoc methods like symlinking a source file. This commit creates the library and populates it with a couple of files that contain the widely-useful portions of pg_dump's dumputils.c file. dumputils.c survives, because it has some stuff that didn't seem appropriate for fe_utils, but it's significantly smaller and is no longer referenced from any other directory. Follow-on patches will move more stuff into fe_utils. The Mkvcbuild.pm hacking here is just a best guess; we'll see how the buildfarm likes it.
2016-03-24 20:55:44 +01:00
#include "fe_utils/string_utils.h"
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
/*
* Variables for mapping DumpId to DumpableObject
*/
static DumpableObject **dumpIdMap = NULL;
static int allocedDumpIds = 0;
static DumpId lastDumpId = 0;
/*
* Variables for mapping CatalogId to DumpableObject
*/
static bool catalogIdMapValid = false;
static DumpableObject **catalogIdMap = NULL;
static int numCatalogIds = 0;
/*
* These variables are static to avoid the notational cruft of having to pass
Handle extension members when first setting object dump flags in pg_dump. pg_dump's original approach to handling extension member objects was to run around and clear (or set) their dump flags rather late in its data collection process. Unfortunately, quite a lot of code expects those flags to be valid before that; which was an entirely reasonable expectation before we added extensions. In particular, this explains Karsten Hilbert's recent report of pg_upgrade failing on a database in which an extension has been installed into the pg_catalog schema. Its objects are initially marked as not-to-be-dumped on the strength of their schema, and later we change them to must-dump because we're doing a binary upgrade of their extension; but we've already skipped essential tasks like making associated DO_SHELL_TYPE objects. To fix, collect extension membership data first, and incorporate it in the initial setting of the dump flags, so that those are once again correct from the get-go. This has the undesirable side effect of slightly lengthening the time taken before pg_dump acquires table locks, but testing suggests that the increase in that window is not very much. Along the way, get rid of ugly special-case logic for deciding whether to dump procedural languages, FDWs, and foreign servers; dump decisions for those are now correct up-front, too. In 9.3 and up, this also fixes erroneous logic about when to dump event triggers (basically, they were *always* dumped before). In 9.5 and up, transform objects had that problem too. Since this problem came in with extensions, back-patch to all supported versions.
2016-01-14 00:55:27 +01:00
* them into findTableByOid() and friends. For each of these arrays, we build
* a sorted-by-OID index array immediately after the objects are fetched,
* and then we use binary search in findTableByOid() and friends. (qsort'ing
* the object arrays themselves would be simpler, but it doesn't work because
* pg_dump.c may have already established pointers between items.)
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
*/
static DumpableObject **tblinfoindex;
static DumpableObject **typinfoindex;
static DumpableObject **funinfoindex;
static DumpableObject **oprinfoindex;
static DumpableObject **collinfoindex;
Use binary search instead of brute-force scan in findNamespace(). The previous coding presented a significant bottleneck when dumping databases containing many thousands of schemas, since the total time spent searching would increase roughly as O(N^2) in the number of objects. Noted by Jeff Janes, though I rewrote his proposed patch to use the existing findObjectByOid infrastructure. Since this is a longstanding performance bug, backpatch to all supported versions.
2012-05-25 20:35:37 +02:00
static DumpableObject **nspinfoindex;
Handle extension members when first setting object dump flags in pg_dump. pg_dump's original approach to handling extension member objects was to run around and clear (or set) their dump flags rather late in its data collection process. Unfortunately, quite a lot of code expects those flags to be valid before that; which was an entirely reasonable expectation before we added extensions. In particular, this explains Karsten Hilbert's recent report of pg_upgrade failing on a database in which an extension has been installed into the pg_catalog schema. Its objects are initially marked as not-to-be-dumped on the strength of their schema, and later we change them to must-dump because we're doing a binary upgrade of their extension; but we've already skipped essential tasks like making associated DO_SHELL_TYPE objects. To fix, collect extension membership data first, and incorporate it in the initial setting of the dump flags, so that those are once again correct from the get-go. This has the undesirable side effect of slightly lengthening the time taken before pg_dump acquires table locks, but testing suggests that the increase in that window is not very much. Along the way, get rid of ugly special-case logic for deciding whether to dump procedural languages, FDWs, and foreign servers; dump decisions for those are now correct up-front, too. In 9.3 and up, this also fixes erroneous logic about when to dump event triggers (basically, they were *always* dumped before). In 9.5 and up, transform objects had that problem too. Since this problem came in with extensions, back-patch to all supported versions.
2016-01-14 00:55:27 +01:00
static DumpableObject **extinfoindex;
static int numTables;
static int numTypes;
static int numFuncs;
static int numOperators;
static int numCollations;
static int numNamespaces;
static int numExtensions;
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
Handle extension members when first setting object dump flags in pg_dump. pg_dump's original approach to handling extension member objects was to run around and clear (or set) their dump flags rather late in its data collection process. Unfortunately, quite a lot of code expects those flags to be valid before that; which was an entirely reasonable expectation before we added extensions. In particular, this explains Karsten Hilbert's recent report of pg_upgrade failing on a database in which an extension has been installed into the pg_catalog schema. Its objects are initially marked as not-to-be-dumped on the strength of their schema, and later we change them to must-dump because we're doing a binary upgrade of their extension; but we've already skipped essential tasks like making associated DO_SHELL_TYPE objects. To fix, collect extension membership data first, and incorporate it in the initial setting of the dump flags, so that those are once again correct from the get-go. This has the undesirable side effect of slightly lengthening the time taken before pg_dump acquires table locks, but testing suggests that the increase in that window is not very much. Along the way, get rid of ugly special-case logic for deciding whether to dump procedural languages, FDWs, and foreign servers; dump decisions for those are now correct up-front, too. In 9.3 and up, this also fixes erroneous logic about when to dump event triggers (basically, they were *always* dumped before). In 9.5 and up, transform objects had that problem too. Since this problem came in with extensions, back-patch to all supported versions.
2016-01-14 00:55:27 +01:00
/* This is an array of object identities, not actual DumpableObjects */
static ExtensionMemberId *extmembers;
static int numextmembers;
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
static void flagInhTables(TableInfo *tbinfo, int numTables,
InhInfo *inhinfo, int numInherits);
pg_dump: Reduce use of global variables Most pg_dump.c global variables, which were passed down individually to dumping routines, are now grouped as members of the new DumpOptions struct, which is used as a local variable and passed down into routines that need it. This helps future development efforts; in particular it is said to enable a mode in which a parallel pg_dump run can output multiple streams, and have them restored in parallel. Also take the opportunity to clean up the pg_dump header files somewhat, to avoid circularity. Author: Joachim Wieland, revised by Álvaro Herrera Reviewed by Peter Eisentraut
2014-10-14 20:00:55 +02:00
static void flagInhAttrs(DumpOptions *dopt, TableInfo *tblinfo, int numTables);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
static DumpableObject **buildIndexArray(void *objArray, int numObjs,
Size objSize);
static int DOCatalogIdCompare(const void *p1, const void *p2);
Handle extension members when first setting object dump flags in pg_dump. pg_dump's original approach to handling extension member objects was to run around and clear (or set) their dump flags rather late in its data collection process. Unfortunately, quite a lot of code expects those flags to be valid before that; which was an entirely reasonable expectation before we added extensions. In particular, this explains Karsten Hilbert's recent report of pg_upgrade failing on a database in which an extension has been installed into the pg_catalog schema. Its objects are initially marked as not-to-be-dumped on the strength of their schema, and later we change them to must-dump because we're doing a binary upgrade of their extension; but we've already skipped essential tasks like making associated DO_SHELL_TYPE objects. To fix, collect extension membership data first, and incorporate it in the initial setting of the dump flags, so that those are once again correct from the get-go. This has the undesirable side effect of slightly lengthening the time taken before pg_dump acquires table locks, but testing suggests that the increase in that window is not very much. Along the way, get rid of ugly special-case logic for deciding whether to dump procedural languages, FDWs, and foreign servers; dump decisions for those are now correct up-front, too. In 9.3 and up, this also fixes erroneous logic about when to dump event triggers (basically, they were *always* dumped before). In 9.5 and up, transform objects had that problem too. Since this problem came in with extensions, back-patch to all supported versions.
2016-01-14 00:55:27 +01:00
static int ExtensionMemberIdCompare(const void *p1, const void *p2);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
static void findParentsByOid(TableInfo *self,
InhInfo *inhinfo, int numInherits);
static int strInArray(const char *pattern, char **arr, int arr_size);
Revise psql pattern-matching switches as per discussion. The rule is now to process all inclusion switches then all exclusion switches, so that the behavior is independent of switch ordering. Use of -T does not cause non-table objects to be suppressed. And the patterns are now interpreted the same way psql's \d commands do it, rather than as pure regex commands; this allows for example -t schema.tab to do what it should have been doing all along. Re-enable the --blobs switch to do something useful, ie, add back blobs into a dump they were otherwise suppressed from.
2006-10-10 01:36:59 +02:00
/*
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
* getSchemaData
* Collect information about all potentially dumpable objects
Revise psql pattern-matching switches as per discussion. The rule is now to process all inclusion switches then all exclusion switches, so that the behavior is independent of switch ordering. Use of -T does not cause non-table objects to be suppressed. And the patterns are now interpreted the same way psql's \d commands do it, rather than as pure regex commands; this allows for example -t schema.tab to do what it should have been doing all along. Re-enable the --blobs switch to do something useful, ie, add back blobs into a dump they were otherwise suppressed from.
2006-10-10 01:36:59 +02:00
*/
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
TableInfo *
Access pg_dump's options structs through Archive struct, not directly. Rather than passing around DumpOptions and RestoreOptions as separate arguments, add fields to struct Archive to carry pointers to these objects, and access them through those fields when needed. There already was a RestoreOptions pointer in Archive, though for no obvious reason it was part of the "private" struct rather than out where pg_dump.c could see it. Doing this allows reversion of quite a lot of parameter-addition changes made in commit 0eea8047bf, which is a good thing IMO because this will reduce the code delta between 9.4 and 9.5, probably easing a few future back-patch efforts. Moreover, the previous commit only added a DumpOptions argument to functions that had to have it at the time, which means we could anticipate still more code churn (and more back-patch hazard) as the requirement spread further. I'd hit exactly that problem in my upcoming patch to fix extension membership marking, which is what motivated me to do this.
2016-01-13 23:48:33 +01:00
getSchemaData(Archive *fout, int *numTablesPtr)
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
{
Handle extension members when first setting object dump flags in pg_dump. pg_dump's original approach to handling extension member objects was to run around and clear (or set) their dump flags rather late in its data collection process. Unfortunately, quite a lot of code expects those flags to be valid before that; which was an entirely reasonable expectation before we added extensions. In particular, this explains Karsten Hilbert's recent report of pg_upgrade failing on a database in which an extension has been installed into the pg_catalog schema. Its objects are initially marked as not-to-be-dumped on the strength of their schema, and later we change them to must-dump because we're doing a binary upgrade of their extension; but we've already skipped essential tasks like making associated DO_SHELL_TYPE objects. To fix, collect extension membership data first, and incorporate it in the initial setting of the dump flags, so that those are once again correct from the get-go. This has the undesirable side effect of slightly lengthening the time taken before pg_dump acquires table locks, but testing suggests that the increase in that window is not very much. Along the way, get rid of ugly special-case logic for deciding whether to dump procedural languages, FDWs, and foreign servers; dump decisions for those are now correct up-front, too. In 9.3 and up, this also fixes erroneous logic about when to dump event triggers (basically, they were *always* dumped before). In 9.5 and up, transform objects had that problem too. Since this problem came in with extensions, back-patch to all supported versions.
2016-01-14 00:55:27 +01:00
TableInfo *tblinfo;
TypeInfo *typinfo;
FuncInfo *funinfo;
OprInfo *oprinfo;
CollInfo *collinfo;
NamespaceInfo *nspinfo;
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
ExtensionInfo *extinfo;
InhInfo *inhinfo;
int numAggregates;
int numInherits;
int numRules;
int numProcLangs;
int numCasts;
Add transforms feature This provides a mechanism for specifying conversions between SQL data types and procedural languages. As examples, there are transforms for hstore and ltree for PL/Perl and PL/Python. reviews by Pavel Stěhule and Andres Freund
2015-04-26 16:33:14 +02:00
int numTransforms;
Support CREATE ACCESS METHOD This enables external code to create access methods. This is useful so that extensions can add their own access methods which can be formally tracked for dependencies, so that DROP operates correctly. Also, having explicit support makes pg_dump work correctly. Currently only index AMs are supported, but we expect different types to be added in the future. Authors: Alexander Korotkov, Petr Jelínek Reviewed-By: Teodor Sigaev, Petr Jelínek, Jim Nasby Commitfest-URL: https://commitfest.postgresql.org/9/353/ Discussion: https://www.postgresql.org/message-id/CAPpHfdsXwZmojm6Dx+TJnpYk27kT4o7Ri6X_4OSWcByu1Rm+VA@mail.gmail.com
2016-03-24 03:01:35 +01:00
int numAccessMethods;
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
int numOpclasses;
int numOpfamilies;
int numConversions;
int numTSParsers;
int numTSTemplates;
int numTSDicts;
int numTSConfigs;
int numForeignDataWrappers;
int numForeignServers;
int numDefaultACLs;
Syntax support and documentation for event triggers. They don't actually do anything yet; that will get fixed in a follow-on commit. But this gets the basic infrastructure in place, including CREATE/ALTER/DROP EVENT TRIGGER; support for COMMENT, SECURITY LABEL, and ALTER EXTENSION .. ADD/DROP EVENT TRIGGER; pg_dump and psql support; and documentation for the anticipated initial feature set. Dimitri Fontaine, with review and a bunch of additional hacking by me. Thom Brown extensively reviewed earlier versions of this patch set, but there's not a whole lot of that code left in this commit, as it turns out.
2012-07-18 16:16:16 +02:00
int numEventTriggers;
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
Handle extension members when first setting object dump flags in pg_dump. pg_dump's original approach to handling extension member objects was to run around and clear (or set) their dump flags rather late in its data collection process. Unfortunately, quite a lot of code expects those flags to be valid before that; which was an entirely reasonable expectation before we added extensions. In particular, this explains Karsten Hilbert's recent report of pg_upgrade failing on a database in which an extension has been installed into the pg_catalog schema. Its objects are initially marked as not-to-be-dumped on the strength of their schema, and later we change them to must-dump because we're doing a binary upgrade of their extension; but we've already skipped essential tasks like making associated DO_SHELL_TYPE objects. To fix, collect extension membership data first, and incorporate it in the initial setting of the dump flags, so that those are once again correct from the get-go. This has the undesirable side effect of slightly lengthening the time taken before pg_dump acquires table locks, but testing suggests that the increase in that window is not very much. Along the way, get rid of ugly special-case logic for deciding whether to dump procedural languages, FDWs, and foreign servers; dump decisions for those are now correct up-front, too. In 9.3 and up, this also fixes erroneous logic about when to dump event triggers (basically, they were *always* dumped before). In 9.5 and up, transform objects had that problem too. Since this problem came in with extensions, back-patch to all supported versions.
2016-01-14 00:55:27 +01:00
/*
* We must read extensions and extension membership info first, because
* extension membership needs to be consultable during decisions about
* whether other objects are to be dumped.
*/
if (g_verbose)
write_msg(NULL, "reading extensions\n");
extinfo = getExtensions(fout, &numExtensions);
extinfoindex = buildIndexArray(extinfo, numExtensions, sizeof(ExtensionInfo));
if (g_verbose)
write_msg(NULL, "identifying extension members\n");
getExtensionMembership(fout, extinfo, numExtensions);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading schemas\n");
Use binary search instead of brute-force scan in findNamespace(). The previous coding presented a significant bottleneck when dumping databases containing many thousands of schemas, since the total time spent searching would increase roughly as O(N^2) in the number of objects. Noted by Jeff Janes, though I rewrote his proposed patch to use the existing findObjectByOid infrastructure. Since this is a longstanding performance bug, backpatch to all supported versions.
2012-05-25 20:35:37 +02:00
nspinfo = getNamespaces(fout, &numNamespaces);
nspinfoindex = buildIndexArray(nspinfo, numNamespaces, sizeof(NamespaceInfo));
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
/*
* getTables should be done as soon as possible, so as to minimize the
* window between starting our transaction and acquiring per-table locks.
* However, we have to do getNamespaces first because the tables get
* linked to their containing namespaces during getTables.
*/
if (g_verbose)
write_msg(NULL, "reading user-defined tables\n");
Access pg_dump's options structs through Archive struct, not directly. Rather than passing around DumpOptions and RestoreOptions as separate arguments, add fields to struct Archive to carry pointers to these objects, and access them through those fields when needed. There already was a RestoreOptions pointer in Archive, though for no obvious reason it was part of the "private" struct rather than out where pg_dump.c could see it. Doing this allows reversion of quite a lot of parameter-addition changes made in commit 0eea8047bf, which is a good thing IMO because this will reduce the code delta between 9.4 and 9.5, probably easing a few future back-patch efforts. Moreover, the previous commit only added a DumpOptions argument to functions that had to have it at the time, which means we could anticipate still more code churn (and more back-patch hazard) as the requirement spread further. I'd hit exactly that problem in my upcoming patch to fix extension membership marking, which is what motivated me to do this.
2016-01-13 23:48:33 +01:00
tblinfo = getTables(fout, &numTables);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
tblinfoindex = buildIndexArray(tblinfo, numTables, sizeof(TableInfo));
Fix O(N^2) behavior in pg_dump for large numbers of owned sequences. The loop that matched owned sequences to their owning tables required time proportional to number of owned sequences times number of tables; although this work was only expended in selective-dump situations, which is probably why the issue wasn't recognized long since. Refactor slightly so that we can perform this work after the index array for findTableByOid has been set up, reducing the time to O(M log N). Per gripe from Mike Roest. Since this is a longstanding performance bug, backpatch to all supported versions.
2012-03-31 20:42:17 +02:00
/* Do this after we've built tblinfoindex */
getOwnedSeqs(fout, tblinfo, numTables);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined functions\n");
Access pg_dump's options structs through Archive struct, not directly. Rather than passing around DumpOptions and RestoreOptions as separate arguments, add fields to struct Archive to carry pointers to these objects, and access them through those fields when needed. There already was a RestoreOptions pointer in Archive, though for no obvious reason it was part of the "private" struct rather than out where pg_dump.c could see it. Doing this allows reversion of quite a lot of parameter-addition changes made in commit 0eea8047bf, which is a good thing IMO because this will reduce the code delta between 9.4 and 9.5, probably easing a few future back-patch efforts. Moreover, the previous commit only added a DumpOptions argument to functions that had to have it at the time, which means we could anticipate still more code churn (and more back-patch hazard) as the requirement spread further. I'd hit exactly that problem in my upcoming patch to fix extension membership marking, which is what motivated me to do this.
2016-01-13 23:48:33 +01:00
funinfo = getFuncs(fout, &numFuncs);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
funinfoindex = buildIndexArray(funinfo, numFuncs, sizeof(FuncInfo));
/* this must be after getTables and getFuncs */
if (g_verbose)
write_msg(NULL, "reading user-defined types\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
typinfo = getTypes(fout, &numTypes);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
typinfoindex = buildIndexArray(typinfo, numTypes, sizeof(TypeInfo));
/* this must be after getFuncs, too */
if (g_verbose)
write_msg(NULL, "reading procedural languages\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getProcLangs(fout, &numProcLangs);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined aggregate functions\n");
Access pg_dump's options structs through Archive struct, not directly. Rather than passing around DumpOptions and RestoreOptions as separate arguments, add fields to struct Archive to carry pointers to these objects, and access them through those fields when needed. There already was a RestoreOptions pointer in Archive, though for no obvious reason it was part of the "private" struct rather than out where pg_dump.c could see it. Doing this allows reversion of quite a lot of parameter-addition changes made in commit 0eea8047bf, which is a good thing IMO because this will reduce the code delta between 9.4 and 9.5, probably easing a few future back-patch efforts. Moreover, the previous commit only added a DumpOptions argument to functions that had to have it at the time, which means we could anticipate still more code churn (and more back-patch hazard) as the requirement spread further. I'd hit exactly that problem in my upcoming patch to fix extension membership marking, which is what motivated me to do this.
2016-01-13 23:48:33 +01:00
getAggregates(fout, &numAggregates);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined operators\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
oprinfo = getOperators(fout, &numOperators);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
oprinfoindex = buildIndexArray(oprinfo, numOperators, sizeof(OprInfo));
Support CREATE ACCESS METHOD This enables external code to create access methods. This is useful so that extensions can add their own access methods which can be formally tracked for dependencies, so that DROP operates correctly. Also, having explicit support makes pg_dump work correctly. Currently only index AMs are supported, but we expect different types to be added in the future. Authors: Alexander Korotkov, Petr Jelínek Reviewed-By: Teodor Sigaev, Petr Jelínek, Jim Nasby Commitfest-URL: https://commitfest.postgresql.org/9/353/ Discussion: https://www.postgresql.org/message-id/CAPpHfdsXwZmojm6Dx+TJnpYk27kT4o7Ri6X_4OSWcByu1Rm+VA@mail.gmail.com
2016-03-24 03:01:35 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined access methods\n");
getAccessMethods(fout, &numAccessMethods);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined operator classes\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getOpclasses(fout, &numOpclasses);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined operator families\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getOpfamilies(fout, &numOpfamilies);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined text search parsers\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getTSParsers(fout, &numTSParsers);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined text search templates\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getTSTemplates(fout, &numTSTemplates);
Revise psql pattern-matching switches as per discussion. The rule is now to process all inclusion switches then all exclusion switches, so that the behavior is independent of switch ordering. Use of -T does not cause non-table objects to be suppressed. And the patterns are now interpreted the same way psql's \d commands do it, rather than as pure regex commands; this allows for example -t schema.tab to do what it should have been doing all along. Re-enable the --blobs switch to do something useful, ie, add back blobs into a dump they were otherwise suppressed from.
2006-10-10 01:36:59 +02:00
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined text search dictionaries\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getTSDictionaries(fout, &numTSDicts);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined text search configurations\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getTSConfigurations(fout, &numTSConfigs);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined foreign-data wrappers\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getForeignDataWrappers(fout, &numForeignDataWrappers);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined foreign servers\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getForeignServers(fout, &numForeignServers);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading default privileges\n");
Access pg_dump's options structs through Archive struct, not directly. Rather than passing around DumpOptions and RestoreOptions as separate arguments, add fields to struct Archive to carry pointers to these objects, and access them through those fields when needed. There already was a RestoreOptions pointer in Archive, though for no obvious reason it was part of the "private" struct rather than out where pg_dump.c could see it. Doing this allows reversion of quite a lot of parameter-addition changes made in commit 0eea8047bf, which is a good thing IMO because this will reduce the code delta between 9.4 and 9.5, probably easing a few future back-patch efforts. Moreover, the previous commit only added a DumpOptions argument to functions that had to have it at the time, which means we could anticipate still more code churn (and more back-patch hazard) as the requirement spread further. I'd hit exactly that problem in my upcoming patch to fix extension membership marking, which is what motivated me to do this.
2016-01-13 23:48:33 +01:00
getDefaultACLs(fout, &numDefaultACLs);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading user-defined collations\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
collinfo = getCollations(fout, &numCollations);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
collinfoindex = buildIndexArray(collinfo, numCollations, sizeof(CollInfo));
if (g_verbose)
write_msg(NULL, "reading user-defined conversions\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getConversions(fout, &numConversions);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading type casts\n");
Access pg_dump's options structs through Archive struct, not directly. Rather than passing around DumpOptions and RestoreOptions as separate arguments, add fields to struct Archive to carry pointers to these objects, and access them through those fields when needed. There already was a RestoreOptions pointer in Archive, though for no obvious reason it was part of the "private" struct rather than out where pg_dump.c could see it. Doing this allows reversion of quite a lot of parameter-addition changes made in commit 0eea8047bf, which is a good thing IMO because this will reduce the code delta between 9.4 and 9.5, probably easing a few future back-patch efforts. Moreover, the previous commit only added a DumpOptions argument to functions that had to have it at the time, which means we could anticipate still more code churn (and more back-patch hazard) as the requirement spread further. I'd hit exactly that problem in my upcoming patch to fix extension membership marking, which is what motivated me to do this.
2016-01-13 23:48:33 +01:00
getCasts(fout, &numCasts);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
Add transforms feature This provides a mechanism for specifying conversions between SQL data types and procedural languages. As examples, there are transforms for hstore and ltree for PL/Perl and PL/Python. reviews by Pavel Stěhule and Andres Freund
2015-04-26 16:33:14 +02:00
if (g_verbose)
write_msg(NULL, "reading transforms\n");
getTransforms(fout, &numTransforms);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading table inheritance information\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
inhinfo = getInherits(fout, &numInherits);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
Fix ordering of obj id for Rules and EventTriggers in pg_dump. getSchemaData() must identify extension member objects and mark them as not to be dumped. This must happen after reading all objects that can be direct members of extensions, but before we begin to process table subsidiary objects. Both rules and event triggers were wrong in this regard. Backport rules portion of patch to 9.1 -- event triggers do not exist prior to 9.3. Suggested fix by Tom Lane, initial complaint and patch by me.
2013-06-10 02:30:39 +02:00
write_msg(NULL, "reading event triggers\n");
getEventTriggers(fout, &numEventTriggers);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
Handle extension members when first setting object dump flags in pg_dump. pg_dump's original approach to handling extension member objects was to run around and clear (or set) their dump flags rather late in its data collection process. Unfortunately, quite a lot of code expects those flags to be valid before that; which was an entirely reasonable expectation before we added extensions. In particular, this explains Karsten Hilbert's recent report of pg_upgrade failing on a database in which an extension has been installed into the pg_catalog schema. Its objects are initially marked as not-to-be-dumped on the strength of their schema, and later we change them to must-dump because we're doing a binary upgrade of their extension; but we've already skipped essential tasks like making associated DO_SHELL_TYPE objects. To fix, collect extension membership data first, and incorporate it in the initial setting of the dump flags, so that those are once again correct from the get-go. This has the undesirable side effect of slightly lengthening the time taken before pg_dump acquires table locks, but testing suggests that the increase in that window is not very much. Along the way, get rid of ugly special-case logic for deciding whether to dump procedural languages, FDWs, and foreign servers; dump decisions for those are now correct up-front, too. In 9.3 and up, this also fixes erroneous logic about when to dump event triggers (basically, they were *always* dumped before). In 9.5 and up, transform objects had that problem too. Since this problem came in with extensions, back-patch to all supported versions.
2016-01-14 00:55:27 +01:00
/* Identify extension configuration tables that should be dumped */
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
Handle extension members when first setting object dump flags in pg_dump. pg_dump's original approach to handling extension member objects was to run around and clear (or set) their dump flags rather late in its data collection process. Unfortunately, quite a lot of code expects those flags to be valid before that; which was an entirely reasonable expectation before we added extensions. In particular, this explains Karsten Hilbert's recent report of pg_upgrade failing on a database in which an extension has been installed into the pg_catalog schema. Its objects are initially marked as not-to-be-dumped on the strength of their schema, and later we change them to must-dump because we're doing a binary upgrade of their extension; but we've already skipped essential tasks like making associated DO_SHELL_TYPE objects. To fix, collect extension membership data first, and incorporate it in the initial setting of the dump flags, so that those are once again correct from the get-go. This has the undesirable side effect of slightly lengthening the time taken before pg_dump acquires table locks, but testing suggests that the increase in that window is not very much. Along the way, get rid of ugly special-case logic for deciding whether to dump procedural languages, FDWs, and foreign servers; dump decisions for those are now correct up-front, too. In 9.3 and up, this also fixes erroneous logic about when to dump event triggers (basically, they were *always* dumped before). In 9.5 and up, transform objects had that problem too. Since this problem came in with extensions, back-patch to all supported versions.
2016-01-14 00:55:27 +01:00
write_msg(NULL, "finding extension tables\n");
processExtensionTables(fout, extinfo, numExtensions);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
/* Link tables to parents, mark parents of target tables interesting */
if (g_verbose)
write_msg(NULL, "finding inheritance relationships\n");
flagInhTables(tblinfo, numTables, inhinfo, numInherits);
if (g_verbose)
write_msg(NULL, "reading column info for interesting tables\n");
Access pg_dump's options structs through Archive struct, not directly. Rather than passing around DumpOptions and RestoreOptions as separate arguments, add fields to struct Archive to carry pointers to these objects, and access them through those fields when needed. There already was a RestoreOptions pointer in Archive, though for no obvious reason it was part of the "private" struct rather than out where pg_dump.c could see it. Doing this allows reversion of quite a lot of parameter-addition changes made in commit 0eea8047bf, which is a good thing IMO because this will reduce the code delta between 9.4 and 9.5, probably easing a few future back-patch efforts. Moreover, the previous commit only added a DumpOptions argument to functions that had to have it at the time, which means we could anticipate still more code churn (and more back-patch hazard) as the requirement spread further. I'd hit exactly that problem in my upcoming patch to fix extension membership marking, which is what motivated me to do this.
2016-01-13 23:48:33 +01:00
getTableAttrs(fout, tblinfo, numTables);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "flagging inherited columns in subtables\n");
Access pg_dump's options structs through Archive struct, not directly. Rather than passing around DumpOptions and RestoreOptions as separate arguments, add fields to struct Archive to carry pointers to these objects, and access them through those fields when needed. There already was a RestoreOptions pointer in Archive, though for no obvious reason it was part of the "private" struct rather than out where pg_dump.c could see it. Doing this allows reversion of quite a lot of parameter-addition changes made in commit 0eea8047bf, which is a good thing IMO because this will reduce the code delta between 9.4 and 9.5, probably easing a few future back-patch efforts. Moreover, the previous commit only added a DumpOptions argument to functions that had to have it at the time, which means we could anticipate still more code churn (and more back-patch hazard) as the requirement spread further. I'd hit exactly that problem in my upcoming patch to fix extension membership marking, which is what motivated me to do this.
2016-01-13 23:48:33 +01:00
flagInhAttrs(fout->dopt, tblinfo, numTables);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading indexes\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getIndexes(fout, tblinfo, numTables);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading constraints\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getConstraints(fout, tblinfo, numTables);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (g_verbose)
write_msg(NULL, "reading triggers\n");
pg_dump: Remove global Archive pointer. Instead, everything that needs the Archive object now gets it as a parameter. This is necessary infrastructure for parallel pg_dump, but is also amply justified by the ugliness of the current code (though a lot more than this is needed to fix that problem).
2012-02-06 20:07:55 +01:00
getTriggers(fout, tblinfo, numTables);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
Syntax support and documentation for event triggers. They don't actually do anything yet; that will get fixed in a follow-on commit. But this gets the basic infrastructure in place, including CREATE/ALTER/DROP EVENT TRIGGER; support for COMMENT, SECURITY LABEL, and ALTER EXTENSION .. ADD/DROP EVENT TRIGGER; pg_dump and psql support; and documentation for the anticipated initial feature set. Dimitri Fontaine, with review and a bunch of additional hacking by me. Thom Brown extensively reviewed earlier versions of this patch set, but there's not a whole lot of that code left in this commit, as it turns out.
2012-07-18 16:16:16 +02:00
if (g_verbose)
Fix ordering of obj id for Rules and EventTriggers in pg_dump. getSchemaData() must identify extension member objects and mark them as not to be dumped. This must happen after reading all objects that can be direct members of extensions, but before we begin to process table subsidiary objects. Both rules and event triggers were wrong in this regard. Backport rules portion of patch to 9.1 -- event triggers do not exist prior to 9.3. Suggested fix by Tom Lane, initial complaint and patch by me.
2013-06-10 02:30:39 +02:00
write_msg(NULL, "reading rewrite rules\n");
getRules(fout, &numRules);
Syntax support and documentation for event triggers. They don't actually do anything yet; that will get fixed in a follow-on commit. But this gets the basic infrastructure in place, including CREATE/ALTER/DROP EVENT TRIGGER; support for COMMENT, SECURITY LABEL, and ALTER EXTENSION .. ADD/DROP EVENT TRIGGER; pg_dump and psql support; and documentation for the anticipated initial feature set. Dimitri Fontaine, with review and a bunch of additional hacking by me. Thom Brown extensively reviewed earlier versions of this patch set, but there's not a whole lot of that code left in this commit, as it turns out.
2012-07-18 16:16:16 +02:00
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
if (g_verbose)
Rename pg_rowsecurity -> pg_policy and other fixes As pointed out by Robert, we should really have named pg_rowsecurity pg_policy, as the objects stored in that catalog are policies. This patch fixes that and updates the column names to start with 'pol' to match the new catalog name. The security consideration for COPY with row level security, also pointed out by Robert, has also been addressed by remembering and re-checking the OID of the relation initially referenced during COPY processing, to make sure it hasn't changed under us by the time we finish planning out the query which has been built. Robert and Alvaro also commented on missing OCLASS and OBJECT entries for POLICY (formerly ROWSECURITY or POLICY, depending) in various places. This patch fixes that too, which also happens to add the ability to COMMENT on policies. In passing, attempt to improve the consistency of messages, comments, and documentation as well. This removes various incarnations of 'row-security', 'row-level security', 'Row-security', etc, in favor of 'policy', 'row level security' or 'row_security' as appropriate. Happy Thanksgiving!
2014-11-27 07:06:36 +01:00
write_msg(NULL, "reading policies\n");
getPolicies(fout, tblinfo, numTables);
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
*numTablesPtr = numTables;
return tblinfo;
}
/* flagInhTables -
* Fill in parent link fields of every target table, and mark
* parents of target tables as interesting
*
* Note that only direct ancestors of targets are marked interesting.
* This is sufficient; we don't much care whether they inherited their
* attributes or not.
*
* modifies tblinfo
*/
static void
flagInhTables(TableInfo *tblinfo, int numTables,
InhInfo *inhinfo, int numInherits)
Revise psql pattern-matching switches as per discussion. The rule is now to process all inclusion switches then all exclusion switches, so that the behavior is independent of switch ordering. Use of -T does not cause non-table objects to be suppressed. And the patterns are now interpreted the same way psql's \d commands do it, rather than as pure regex commands; this allows for example -t schema.tab to do what it should have been doing all along. Re-enable the --blobs switch to do something useful, ie, add back blobs into a dump they were otherwise suppressed from.
2006-10-10 01:36:59 +02:00
{
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
int i,
j;
int numParents;
TableInfo **parents;
for (i = 0; i < numTables; i++)
{
Update messages, comments and documentation for materialized views. All instances of the verbiage lagging the code. Back-patch to 9.3, where materialized views were introduced.
2013-07-05 21:25:51 +02:00
/* Some kinds never have parents */
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (tblinfo[i].relkind == RELKIND_SEQUENCE ||
Add a materialized view relations. A materialized view has a rule just like a view and a heap and other physical properties like a table. The rule is only used to populate the table, references in queries refer to the materialized data. This is a minimal implementation, but should still be useful in many cases. Currently data is only populated "on demand" by the CREATE MATERIALIZED VIEW and REFRESH MATERIALIZED VIEW statements. It is expected that future releases will add incremental updates with various timings, and that a more refined concept of defining what is "fresh" data will be developed. At some point it may even be possible to have queries use a materialized in place of references to underlying tables, but that requires the other above-mentioned features to be working first. Much of the documentation work by Robert Haas. Review by Noah Misch, Thom Brown, Robert Haas, Marko Tiikkaja Security review by KaiGai Kohei, with a decision on how best to implement sepgsql still pending.
2013-03-04 01:23:31 +01:00
tblinfo[i].relkind == RELKIND_VIEW ||
tblinfo[i].relkind == RELKIND_MATVIEW)
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
continue;
/* Don't bother computing anything for non-target tables, either */
if (!tblinfo[i].dobj.dump)
continue;
/* Find all the immediate parent tables */
findParentsByOid(&tblinfo[i], inhinfo, numInherits);
Revise psql pattern-matching switches as per discussion. The rule is now to process all inclusion switches then all exclusion switches, so that the behavior is independent of switch ordering. Use of -T does not cause non-table objects to be suppressed. And the patterns are now interpreted the same way psql's \d commands do it, rather than as pure regex commands; this allows for example -t schema.tab to do what it should have been doing all along. Re-enable the --blobs switch to do something useful, ie, add back blobs into a dump they were otherwise suppressed from.
2006-10-10 01:36:59 +02:00
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
/* Mark the parents as interesting for getTableAttrs */
numParents = tblinfo[i].numParents;
parents = tblinfo[i].parents;
for (j = 0; j < numParents; j++)
parents[j]->interesting = true;
}
Revise psql pattern-matching switches as per discussion. The rule is now to process all inclusion switches then all exclusion switches, so that the behavior is independent of switch ordering. Use of -T does not cause non-table objects to be suppressed. And the patterns are now interpreted the same way psql's \d commands do it, rather than as pure regex commands; this allows for example -t schema.tab to do what it should have been doing all along. Re-enable the --blobs switch to do something useful, ie, add back blobs into a dump they were otherwise suppressed from.
2006-10-10 01:36:59 +02:00
}
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
/* flagInhAttrs -
* for each dumpable table in tblinfo, flag its inherited attributes
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
*
* What we need to do here is detect child columns that inherit NOT NULL
* bits from their parents (so that we needn't specify that again for the
* child) and child columns that have DEFAULT NULL when their parents had
* some non-null default. In the latter case, we make up a dummy AttrDefInfo
* object so that we'll correctly emit the necessary DEFAULT NULL clause;
* otherwise the backend will apply an inherited default to the column.
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
*
* modifies tblinfo
*/
static void
pg_dump: Reduce use of global variables Most pg_dump.c global variables, which were passed down individually to dumping routines, are now grouped as members of the new DumpOptions struct, which is used as a local variable and passed down into routines that need it. This helps future development efforts; in particular it is said to enable a mode in which a parallel pg_dump run can output multiple streams, and have them restored in parallel. Also take the opportunity to clean up the pg_dump header files somewhat, to avoid circularity. Author: Joachim Wieland, revised by Álvaro Herrera Reviewed by Peter Eisentraut
2014-10-14 20:00:55 +02:00
flagInhAttrs(DumpOptions *dopt, TableInfo *tblinfo, int numTables)
Revise psql pattern-matching switches as per discussion. The rule is now to process all inclusion switches then all exclusion switches, so that the behavior is independent of switch ordering. Use of -T does not cause non-table objects to be suppressed. And the patterns are now interpreted the same way psql's \d commands do it, rather than as pure regex commands; this allows for example -t schema.tab to do what it should have been doing all along. Re-enable the --blobs switch to do something useful, ie, add back blobs into a dump they were otherwise suppressed from.
2006-10-10 01:36:59 +02:00
{
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
int i,
j,
k;
for (i = 0; i < numTables; i++)
{
TableInfo *tbinfo = &(tblinfo[i]);
int numParents;
TableInfo **parents;
Update messages, comments and documentation for materialized views. All instances of the verbiage lagging the code. Back-patch to 9.3, where materialized views were introduced.
2013-07-05 21:25:51 +02:00
/* Some kinds never have parents */
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
if (tbinfo->relkind == RELKIND_SEQUENCE ||
Add a materialized view relations. A materialized view has a rule just like a view and a heap and other physical properties like a table. The rule is only used to populate the table, references in queries refer to the materialized data. This is a minimal implementation, but should still be useful in many cases. Currently data is only populated "on demand" by the CREATE MATERIALIZED VIEW and REFRESH MATERIALIZED VIEW statements. It is expected that future releases will add incremental updates with various timings, and that a more refined concept of defining what is "fresh" data will be developed. At some point it may even be possible to have queries use a materialized in place of references to underlying tables, but that requires the other above-mentioned features to be working first. Much of the documentation work by Robert Haas. Review by Noah Misch, Thom Brown, Robert Haas, Marko Tiikkaja Security review by KaiGai Kohei, with a decision on how best to implement sepgsql still pending.
2013-03-04 01:23:31 +01:00
tbinfo->relkind == RELKIND_VIEW ||
tbinfo->relkind == RELKIND_MATVIEW)
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
continue;
/* Don't bother computing anything for non-target tables, either */
if (!tbinfo->dobj.dump)
continue;
numParents = tbinfo->numParents;
parents = tbinfo->parents;
if (numParents == 0)
continue; /* nothing to see here, move along */
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
/* For each column, search for matching column names in parent(s) */
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
for (j = 0; j < tbinfo->numatts; j++)
{
bool foundNotNull; /* Attr was NOT NULL in a parent */
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
bool foundDefault; /* Found a default in a parent */
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
/* no point in examining dropped columns */
if (tbinfo->attisdropped[j])
continue;
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
foundNotNull = false;
foundDefault = false;
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
for (k = 0; k < numParents; k++)
{
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
TableInfo *parent = parents[k];
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
int inhAttrInd;
inhAttrInd = strInArray(tbinfo->attnames[j],
parent->attnames,
parent->numatts);
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
if (inhAttrInd >= 0)
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
{
foundNotNull |= parent->notnull[inhAttrInd];
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
foundDefault |= (parent->attrdefs[inhAttrInd] != NULL);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
}
}
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
/* Remember if we found inherited NOT NULL */
tbinfo->inhNotNull[j] = foundNotNull;
/* Manufacture a DEFAULT NULL clause if necessary */
if (foundDefault && tbinfo->attrdefs[j] == NULL)
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
{
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
AttrDefInfo *attrDef;
attrDef = (AttrDefInfo *) pg_malloc(sizeof(AttrDefInfo));
attrDef->dobj.objType = DO_ATTRDEF;
attrDef->dobj.catId.tableoid = 0;
attrDef->dobj.catId.oid = 0;
AssignDumpId(&attrDef->dobj);
attrDef->dobj.name = pg_strdup(tbinfo->dobj.name);
attrDef->dobj.namespace = tbinfo->dobj.namespace;
attrDef->dobj.dump = tbinfo->dobj.dump;
attrDef->adtable = tbinfo;
attrDef->adnum = j + 1;
attrDef->adef_expr = pg_strdup("NULL");
/* Will column be dumped explicitly? */
pg_dump: Reduce use of global variables Most pg_dump.c global variables, which were passed down individually to dumping routines, are now grouped as members of the new DumpOptions struct, which is used as a local variable and passed down into routines that need it. This helps future development efforts; in particular it is said to enable a mode in which a parallel pg_dump run can output multiple streams, and have them restored in parallel. Also take the opportunity to clean up the pg_dump header files somewhat, to avoid circularity. Author: Joachim Wieland, revised by Álvaro Herrera Reviewed by Peter Eisentraut
2014-10-14 20:00:55 +02:00
if (shouldPrintColumn(dopt, tbinfo, j))
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
{
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
attrDef->separate = false;
/* No dependency needed: NULL cannot have dependencies */
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
}
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
else
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
{
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
/* column will be suppressed, print default separately */
attrDef->separate = true;
/* ensure it comes out after the table */
addObjectDependency(&attrDef->dobj,
tbinfo->dobj.dumpId);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
}
Fix pg_dump for better handling of inherited columns. Revise pg_dump's handling of inherited columns, which was last looked at seriously in 2001, to eliminate several misbehaviors associated with inherited default expressions and NOT NULL flags. In particular make sure that a column is printed in a child table's CREATE TABLE command if and only if it has attislocal = true; the former behavior would sometimes cause a column to become marked attislocal when it was not so marked in the source database. Also, stop relying on textual comparison of default expressions to decide if they're inherited; instead, don't use default-expression inheritance at all, but just install the default explicitly at each level of the hierarchy. This fixes the search-path-related misbehavior recently exhibited by Chester Young, and also removes some dubious assumptions about the order in which ALTER TABLE SET DEFAULT commands would be executed. Back-patch to all supported branches.
2012-02-10 19:28:05 +01:00
tbinfo->attrdefs[j] = attrDef;
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
}
}
}
}
/*
* AssignDumpId
* Given a newly-created dumpable object, assign a dump ID,
* and enter the object into the lookup table.
*
* The caller is expected to have filled in objType and catId,
* but not any of the other standard fields of a DumpableObject.
*/
void
AssignDumpId(DumpableObject *dobj)
{
dobj->dumpId = ++lastDumpId;
dobj->name = NULL; /* must be set later */
dobj->namespace = NULL; /* may be set later */
dobj->dump = true; /* default assumption */
dobj->ext_member = false; /* default assumption */
dobj->dependencies = NULL;
dobj->nDeps = 0;
dobj->allocDeps = 0;
while (dobj->dumpId >= allocedDumpIds)
{
int newAlloc;
if (allocedDumpIds <= 0)
{
newAlloc = 256;
dumpIdMap = (DumpableObject **)
pg_malloc(newAlloc * sizeof(DumpableObject *));
}
else
{
newAlloc = allocedDumpIds * 2;
dumpIdMap = (DumpableObject **)
pg_realloc(dumpIdMap, newAlloc * sizeof(DumpableObject *));
}
memset(dumpIdMap + allocedDumpIds, 0,
(newAlloc - allocedDumpIds) * sizeof(DumpableObject *));
allocedDumpIds = newAlloc;
}
dumpIdMap[dobj->dumpId] = dobj;
/* mark catalogIdMap invalid, but don't rebuild it yet */
catalogIdMapValid = false;
}
/*
* Assign a DumpId that's not tied to a DumpableObject.
*
* This is used when creating a "fixed" ArchiveEntry that doesn't need to
* participate in the sorting logic.
*/
DumpId
createDumpId(void)
{
return ++lastDumpId;
}
/*
* Return the largest DumpId so far assigned
*/
DumpId
getMaxDumpId(void)
{
return lastDumpId;
}
/*
* Find a DumpableObject by dump ID
*
* Returns NULL for invalid ID
*/
DumpableObject *
findObjectByDumpId(DumpId dumpId)
{
if (dumpId <= 0 || dumpId >= allocedDumpIds)
return NULL; /* out of range? */
return dumpIdMap[dumpId];
}
/*
* Find a DumpableObject by catalog ID
*
* Returns NULL for unknown ID
*
* We use binary search in a sorted list that is built on first call.
* If AssignDumpId() and findObjectByCatalogId() calls were freely intermixed,
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* the code would work, but possibly be very slow. In the current usage
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
* pattern that does not happen, indeed we build the list at most twice.
*/
DumpableObject *
findObjectByCatalogId(CatalogId catalogId)
{
DumpableObject **low;
DumpableObject **high;
if (!catalogIdMapValid)
{
if (catalogIdMap)
free(catalogIdMap);
getDumpableObjects(&catalogIdMap, &numCatalogIds);
if (numCatalogIds > 1)
qsort((void *) catalogIdMap, numCatalogIds,
sizeof(DumpableObject *), DOCatalogIdCompare);
catalogIdMapValid = true;
}
/*
* We could use bsearch() here, but the notational cruft of calling
* bsearch is nearly as bad as doing it ourselves; and the generalized
* bsearch function is noticeably slower as well.
*/
if (numCatalogIds <= 0)
return NULL;
low = catalogIdMap;
high = catalogIdMap + (numCatalogIds - 1);
while (low <= high)
{
DumpableObject **middle;
int difference;
middle = low + (high - low) / 2;
/* comparison must match DOCatalogIdCompare, below */
difference = oidcmp((*middle)->catId.oid, catalogId.oid);
if (difference == 0)
difference = oidcmp((*middle)->catId.tableoid, catalogId.tableoid);
if (difference == 0)
return *middle;
else if (difference < 0)
low = middle + 1;
else
high = middle - 1;
}
return NULL;
}
/*
* Find a DumpableObject by OID, in a pre-sorted array of one type of object
*
* Returns NULL for unknown OID
*/
static DumpableObject *
findObjectByOid(Oid oid, DumpableObject **indexArray, int numObjs)
{
DumpableObject **low;
DumpableObject **high;
/*
* This is the same as findObjectByCatalogId except we assume we need not
* look at table OID because the objects are all the same type.
*
* We could use bsearch() here, but the notational cruft of calling
* bsearch is nearly as bad as doing it ourselves; and the generalized
* bsearch function is noticeably slower as well.
*/
if (numObjs <= 0)
return NULL;
low = indexArray;
high = indexArray + (numObjs - 1);
while (low <= high)
{
DumpableObject **middle;
int difference;
middle = low + (high - low) / 2;
difference = oidcmp((*middle)->catId.oid, oid);
if (difference == 0)
return *middle;
else if (difference < 0)
low = middle + 1;
else
high = middle - 1;
}
return NULL;
}
/*
* Build an index array of DumpableObject pointers, sorted by OID
*/
static DumpableObject **
buildIndexArray(void *objArray, int numObjs, Size objSize)
{
DumpableObject **ptrs;
int i;
ptrs = (DumpableObject **) pg_malloc(numObjs * sizeof(DumpableObject *));
for (i = 0; i < numObjs; i++)
ptrs[i] = (DumpableObject *) ((char *) objArray + i * objSize);
/* We can use DOCatalogIdCompare to sort since its first key is OID */
if (numObjs > 1)
qsort((void *) ptrs, numObjs, sizeof(DumpableObject *),
DOCatalogIdCompare);
return ptrs;
}
/*
* qsort comparator for pointers to DumpableObjects
*/
static int
DOCatalogIdCompare(const void *p1, const void *p2)
{
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
const DumpableObject *obj1 = *(DumpableObject *const *) p1;
const DumpableObject *obj2 = *(DumpableObject *const *) p2;
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
int cmpval;
/*
* Compare OID first since it's usually unique, whereas there will only be
* a few distinct values of tableoid.
*/
cmpval = oidcmp(obj1->catId.oid, obj2->catId.oid);
if (cmpval == 0)
cmpval = oidcmp(obj1->catId.tableoid, obj2->catId.tableoid);
return cmpval;
}
/*
* Build an array of pointers to all known dumpable objects
*
* This simply creates a modifiable copy of the internal map.
*/
void
getDumpableObjects(DumpableObject ***objs, int *numObjs)
{
int i,
j;
*objs = (DumpableObject **)
pg_malloc(allocedDumpIds * sizeof(DumpableObject *));
j = 0;
for (i = 1; i < allocedDumpIds; i++)
{
if (dumpIdMap[i])
(*objs)[j++] = dumpIdMap[i];
}
*numObjs = j;
}
/*
* Add a dependency link to a DumpableObject
*
* Note: duplicate dependencies are currently not eliminated
*/
void
addObjectDependency(DumpableObject *dobj, DumpId refId)
{
if (dobj->nDeps >= dobj->allocDeps)
{
if (dobj->allocDeps <= 0)
{
dobj->allocDeps = 16;
dobj->dependencies = (DumpId *)
pg_malloc(dobj->allocDeps * sizeof(DumpId));
}
else
{
dobj->allocDeps *= 2;
dobj->dependencies = (DumpId *)
pg_realloc(dobj->dependencies,
dobj->allocDeps * sizeof(DumpId));
}
}
dobj->dependencies[dobj->nDeps++] = refId;
}
/*
* Remove a dependency link from a DumpableObject
*
* If there are multiple links, all are removed
*/
void
removeObjectDependency(DumpableObject *dobj, DumpId refId)
{
int i;
int j = 0;
for (i = 0; i < dobj->nDeps; i++)
{
if (dobj->dependencies[i] != refId)
dobj->dependencies[j++] = dobj->dependencies[i];
}
dobj->nDeps = j;
}
/*
* findTableByOid
* finds the entry (in tblinfo) of the table with the given oid
* returns NULL if not found
*/
TableInfo *
findTableByOid(Oid oid)
{
return (TableInfo *) findObjectByOid(oid, tblinfoindex, numTables);
}
/*
* findTypeByOid
* finds the entry (in typinfo) of the type with the given oid
* returns NULL if not found
*/
TypeInfo *
findTypeByOid(Oid oid)
{
return (TypeInfo *) findObjectByOid(oid, typinfoindex, numTypes);
}
/*
* findFuncByOid
* finds the entry (in funinfo) of the function with the given oid
* returns NULL if not found
*/
FuncInfo *
findFuncByOid(Oid oid)
{
return (FuncInfo *) findObjectByOid(oid, funinfoindex, numFuncs);
}
/*
* findOprByOid
* finds the entry (in oprinfo) of the operator with the given oid
* returns NULL if not found
*/
OprInfo *
findOprByOid(Oid oid)
{
return (OprInfo *) findObjectByOid(oid, oprinfoindex, numOperators);
}
/*
* findCollationByOid
* finds the entry (in collinfo) of the collation with the given oid
* returns NULL if not found
*/
CollInfo *
findCollationByOid(Oid oid)
{
return (CollInfo *) findObjectByOid(oid, collinfoindex, numCollations);
}
Use binary search instead of brute-force scan in findNamespace(). The previous coding presented a significant bottleneck when dumping databases containing many thousands of schemas, since the total time spent searching would increase roughly as O(N^2) in the number of objects. Noted by Jeff Janes, though I rewrote his proposed patch to use the existing findObjectByOid infrastructure. Since this is a longstanding performance bug, backpatch to all supported versions.
2012-05-25 20:35:37 +02:00
/*
* findNamespaceByOid
* finds the entry (in nspinfo) of the namespace with the given oid
* returns NULL if not found
*/
NamespaceInfo *
findNamespaceByOid(Oid oid)
{
return (NamespaceInfo *) findObjectByOid(oid, nspinfoindex, numNamespaces);
}
Handle extension members when first setting object dump flags in pg_dump. pg_dump's original approach to handling extension member objects was to run around and clear (or set) their dump flags rather late in its data collection process. Unfortunately, quite a lot of code expects those flags to be valid before that; which was an entirely reasonable expectation before we added extensions. In particular, this explains Karsten Hilbert's recent report of pg_upgrade failing on a database in which an extension has been installed into the pg_catalog schema. Its objects are initially marked as not-to-be-dumped on the strength of their schema, and later we change them to must-dump because we're doing a binary upgrade of their extension; but we've already skipped essential tasks like making associated DO_SHELL_TYPE objects. To fix, collect extension membership data first, and incorporate it in the initial setting of the dump flags, so that those are once again correct from the get-go. This has the undesirable side effect of slightly lengthening the time taken before pg_dump acquires table locks, but testing suggests that the increase in that window is not very much. Along the way, get rid of ugly special-case logic for deciding whether to dump procedural languages, FDWs, and foreign servers; dump decisions for those are now correct up-front, too. In 9.3 and up, this also fixes erroneous logic about when to dump event triggers (basically, they were *always* dumped before). In 9.5 and up, transform objects had that problem too. Since this problem came in with extensions, back-patch to all supported versions.
2016-01-14 00:55:27 +01:00
/*
* findExtensionByOid
* finds the entry (in extinfo) of the extension with the given oid
* returns NULL if not found
*/
ExtensionInfo *
findExtensionByOid(Oid oid)
{
return (ExtensionInfo *) findObjectByOid(oid, extinfoindex, numExtensions);
}
/*
* setExtensionMembership
* accept and save data about which objects belong to extensions
*/
void
setExtensionMembership(ExtensionMemberId *extmems, int nextmems)
{
/* Sort array in preparation for binary searches */
if (nextmems > 1)
qsort((void *) extmems, nextmems, sizeof(ExtensionMemberId),
ExtensionMemberIdCompare);
/* And save */
extmembers = extmems;
numextmembers = nextmems;
}
/*
* findOwningExtension
* return owning extension for specified catalog ID, or NULL if none
*/
ExtensionInfo *
findOwningExtension(CatalogId catalogId)
{
ExtensionMemberId *low;
ExtensionMemberId *high;
/*
* We could use bsearch() here, but the notational cruft of calling
* bsearch is nearly as bad as doing it ourselves; and the generalized
* bsearch function is noticeably slower as well.
*/
if (numextmembers <= 0)
return NULL;
low = extmembers;
high = extmembers + (numextmembers - 1);
while (low <= high)
{
ExtensionMemberId *middle;
int difference;
middle = low + (high - low) / 2;
/* comparison must match ExtensionMemberIdCompare, below */
difference = oidcmp(middle->catId.oid, catalogId.oid);
if (difference == 0)
difference = oidcmp(middle->catId.tableoid, catalogId.tableoid);
if (difference == 0)
return middle->ext;
else if (difference < 0)
low = middle + 1;
else
high = middle - 1;
}
return NULL;
}
/*
* qsort comparator for ExtensionMemberIds
*/
static int
ExtensionMemberIdCompare(const void *p1, const void *p2)
{
const ExtensionMemberId *obj1 = (const ExtensionMemberId *) p1;
const ExtensionMemberId *obj2 = (const ExtensionMemberId *) p2;
int cmpval;
/*
* Compare OID first since it's usually unique, whereas there will only be
* a few distinct values of tableoid.
*/
cmpval = oidcmp(obj1->catId.oid, obj2->catId.oid);
if (cmpval == 0)
cmpval = oidcmp(obj1->catId.tableoid, obj2->catId.tableoid);
return cmpval;
}
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
/*
* findParentsByOid
* find a table's parents in tblinfo[]
*/
static void
findParentsByOid(TableInfo *self,
InhInfo *inhinfo, int numInherits)
{
Oid oid = self->dobj.catId.oid;
int i,
j;
int numParents;
numParents = 0;
for (i = 0; i < numInherits; i++)
{
if (inhinfo[i].inhrelid == oid)
numParents++;
}
self->numParents = numParents;
if (numParents > 0)
{
self->parents = (TableInfo **)
pg_malloc(sizeof(TableInfo *) * numParents);
j = 0;
for (i = 0; i < numInherits; i++)
{
if (inhinfo[i].inhrelid == oid)
{
TableInfo *parent;
parent = findTableByOid(inhinfo[i].inhparent);
if (parent == NULL)
{
write_msg(NULL, "failed sanity check, parent OID %u of table \"%s\" (OID %u) not found\n",
inhinfo[i].inhparent,
self->dobj.name,
oid);
Invent on_exit_nicely for pg_dump. Per recent discussions on pgsql-hackers regarding parallel pg_dump.
2012-02-16 17:49:20 +01:00
exit_nicely(1);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
}
self->parents[j++] = parent;
}
}
}
else
self->parents = NULL;
}
/*
* parseOidArray
* parse a string of numbers delimited by spaces into a character array
*
* Note: actually this is used for both Oids and potentially-signed
* attribute numbers. This should cause no trouble, but we could split
* the function into two functions with different argument types if it does.
*/
void
parseOidArray(const char *str, Oid *array, int arraysize)
{
int j,
argNum;
char temp[100];
char s;
argNum = 0;
j = 0;
for (;;)
{
s = *str++;
if (s == ' ' || s == '\0')
{
if (j > 0)
{
if (argNum >= arraysize)
{
write_msg(NULL, "could not parse numeric array \"%s\": too many numbers\n", str);
Invent on_exit_nicely for pg_dump. Per recent discussions on pgsql-hackers regarding parallel pg_dump.
2012-02-16 17:49:20 +01:00
exit_nicely(1);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
}
temp[j] = '\0';
array[argNum++] = atooid(temp);
j = 0;
}
if (s == '\0')
break;
}
else
{
if (!(isdigit((unsigned char) s) || s == '-') ||
j >= sizeof(temp) - 1)
{
write_msg(NULL, "could not parse numeric array \"%s\": invalid character in number\n", str);
Invent on_exit_nicely for pg_dump. Per recent discussions on pgsql-hackers regarding parallel pg_dump.
2012-02-16 17:49:20 +01:00
exit_nicely(1);
Move pg_dump memory routines into pg_dumpmem.c/h and restore common.c with its original functions. The previous function migration would cause too many difficulties in back-patching.
2011-11-27 04:34:36 +01:00
}
temp[j++] = s;
}
}
while (argNum < arraysize)
array[argNum++] = InvalidOid;
}
/*
* strInArray:
* takes in a string and a string array and the number of elements in the
* string array.
* returns the index if the string is somewhere in the array, -1 otherwise
*/
static int
strInArray(const char *pattern, char **arr, int arr_size)
{
int i;
for (i = 0; i < arr_size; i++)
{
if (strcmp(pattern, arr[i]) == 0)
return i;
}
return -1;
}