rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/bin/pg_dump/t/002_pg_dump.pl

3446 lines
100 KiB
Perl
Raw Normal View History

Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
use strict;
use warnings;
use Config;
use PostgresNode;
use TestLib;
use Test::More;
my $tempdir = TestLib::tempdir;
my $tempdir_short = TestLib::tempdir_short;
###############################################################
# Definition of the pg_dump runs to make.
#
# Each of these runs are named and those names are used below
# to define how each test should (or shouldn't) treat a result
# from a given run.
#
# test_key indicates that a given run should simply use the same
# set of like/unlike tests as another run, and which run that is.
#
# dump_cmd is the pg_dump command to run, which is an array of
# the full command and arguments to run. Note that this is run
# using $node->command_ok(), so the port does not need to be
# specified and is pulled from $PGPORT, which is set by the
# PostgresNode system.
#
# restore_cmd is the pg_restore command to run, if any. Note
# that this should generally be used when the pg_dump goes to
# a non-text file and that the restore can then be used to
# generate a text file to run through the tests from the
# non-text file generated by pg_dump.
#
# TODO: Have pg_restore actually restore to an independent
# database and then pg_dump *that* database (or something along
# those lines) to validate that part of the process.
my %pgdump_runs = (
binary_upgrade => {
dump_cmd => [
'pg_dump',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
'--no-sync',
pg_upgrade: Fix large object COMMENTS, SECURITY LABELS When performing a pg_upgrade, we copy the files behind pg_largeobject and pg_largeobject_metadata, allowing us to avoid having to dump out and reload the actual data for large objects and their ACLs. Unfortunately, that isn't all of the information which can be associated with large objects. Currently, we also support COMMENTs and SECURITY LABELs with large objects and these were being silently dropped during a pg_upgrade as pg_dump would skip everything having to do with a large object and pg_upgrade only copied the tables mentioned to the new cluster. As the file copies happen after the catalog dump and reload, we can't simply include the COMMENTs and SECURITY LABELs in pg_dump's binary-mode output but we also have to include the actual large object definition as well. With the definition, comments, and security labels in the pg_dump output and the file copies performed by pg_upgrade, all of the data and metadata associated with large objects is able to be successfully pulled forward across a pg_upgrade. In 9.6 and master, we can simply adjust the dump bitmask to indicate which components we don't want. In 9.5 and earlier, we have to put explciit checks in in dumpBlob() and dumpBlobs() to not include the ACL or the data when in binary-upgrade mode. Adjustments made to the privileges regression test to allow another test (large_object.sql) to be added which explicitly leaves a large object with a comment in place to provide coverage of that case with pg_upgrade. Back-patch to all supported branches. Discussion: https://postgr.es/m/20170221162655.GE9812@tamriel.snowman.net
2017-03-06 23:03:57 +01:00
'--format=custom',
"--file=$tempdir/binary_upgrade.dump",
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'-w',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'--schema-only',
'--binary-upgrade',
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
'-d', 'postgres', # alternative way to specify database
pg_upgrade: Fix large object COMMENTS, SECURITY LABELS When performing a pg_upgrade, we copy the files behind pg_largeobject and pg_largeobject_metadata, allowing us to avoid having to dump out and reload the actual data for large objects and their ACLs. Unfortunately, that isn't all of the information which can be associated with large objects. Currently, we also support COMMENTs and SECURITY LABELs with large objects and these were being silently dropped during a pg_upgrade as pg_dump would skip everything having to do with a large object and pg_upgrade only copied the tables mentioned to the new cluster. As the file copies happen after the catalog dump and reload, we can't simply include the COMMENTs and SECURITY LABELs in pg_dump's binary-mode output but we also have to include the actual large object definition as well. With the definition, comments, and security labels in the pg_dump output and the file copies performed by pg_upgrade, all of the data and metadata associated with large objects is able to be successfully pulled forward across a pg_upgrade. In 9.6 and master, we can simply adjust the dump bitmask to indicate which components we don't want. In 9.5 and earlier, we have to put explciit checks in in dumpBlob() and dumpBlobs() to not include the ACL or the data when in binary-upgrade mode. Adjustments made to the privileges regression test to allow another test (large_object.sql) to be added which explicitly leaves a large object with a comment in place to provide coverage of that case with pg_upgrade. Back-patch to all supported branches. Discussion: https://postgr.es/m/20170221162655.GE9812@tamriel.snowman.net
2017-03-06 23:03:57 +01:00
],
restore_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_restore', '-Fc', '--verbose',
pg_upgrade: Fix large object COMMENTS, SECURITY LABELS When performing a pg_upgrade, we copy the files behind pg_largeobject and pg_largeobject_metadata, allowing us to avoid having to dump out and reload the actual data for large objects and their ACLs. Unfortunately, that isn't all of the information which can be associated with large objects. Currently, we also support COMMENTs and SECURITY LABELs with large objects and these were being silently dropped during a pg_upgrade as pg_dump would skip everything having to do with a large object and pg_upgrade only copied the tables mentioned to the new cluster. As the file copies happen after the catalog dump and reload, we can't simply include the COMMENTs and SECURITY LABELs in pg_dump's binary-mode output but we also have to include the actual large object definition as well. With the definition, comments, and security labels in the pg_dump output and the file copies performed by pg_upgrade, all of the data and metadata associated with large objects is able to be successfully pulled forward across a pg_upgrade. In 9.6 and master, we can simply adjust the dump bitmask to indicate which components we don't want. In 9.5 and earlier, we have to put explciit checks in in dumpBlob() and dumpBlobs() to not include the ACL or the data when in binary-upgrade mode. Adjustments made to the privileges regression test to allow another test (large_object.sql) to be added which explicitly leaves a large object with a comment in place to provide coverage of that case with pg_upgrade. Back-patch to all supported branches. Discussion: https://postgr.es/m/20170221162655.GE9812@tamriel.snowman.net
2017-03-06 23:03:57 +01:00
"--file=$tempdir/binary_upgrade.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
"$tempdir/binary_upgrade.dump",
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
clean => {
dump_cmd => [
'pg_dump',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
'--no-sync',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
"--file=$tempdir/clean.sql",
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'-c',
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
'-d', 'postgres', # alternative way to specify database
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
clean_if_exists => {
dump_cmd => [
'pg_dump',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
'--no-sync',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
"--file=$tempdir/clean_if_exists.sql",
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'-c',
'--if-exists',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'--encoding=UTF8', # no-op, just tests that option is accepted
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
column_inserts => {
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dump', '--no-sync',
"--file=$tempdir/column_inserts.sql", '-a',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'--column-inserts', 'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
createdb => {
dump_cmd => [
'pg_dump',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
'--no-sync',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
"--file=$tempdir/createdb.sql",
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'-C',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'-R', # no-op, just for testing
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'-v',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
data_only => {
dump_cmd => [
'pg_dump',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
'--no-sync',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
"--file=$tempdir/data_only.sql",
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'-a',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'--superuser=test_superuser',
'--disable-triggers',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'-v', # no-op, just make sure it works
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
defaults => {
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dump', '--no-sync',
'-f', "$tempdir/defaults.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
defaults_no_public => {
database => 'regress_pg_dump_test',
dump_cmd => [
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
'pg_dump', '--no-sync', '-f', "$tempdir/defaults_no_public.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'regress_pg_dump_test',
],
},
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
defaults_no_public_clean => {
database => 'regress_pg_dump_test',
dump_cmd => [
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
'pg_dump', '--no-sync', '-c', '-f',
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
"$tempdir/defaults_no_public_clean.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'regress_pg_dump_test',
],
},
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
# Do not use --no-sync to give test coverage for data sync.
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
defaults_custom_format => {
test_key => 'defaults',
dump_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_dump', '-Fc', '-Z6',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
"--file=$tempdir/defaults_custom_format.dump", 'postgres',
],
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
restore_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_restore', '-Fc',
"--file=$tempdir/defaults_custom_format.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
"$tempdir/defaults_custom_format.dump",
],
},
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
# Do not use --no-sync to give test coverage for data sync.
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
defaults_dir_format => {
test_key => 'defaults',
dump_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_dump', '-Fd',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
"--file=$tempdir/defaults_dir_format", 'postgres',
],
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
restore_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_restore', '-Fd',
"--file=$tempdir/defaults_dir_format.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
"$tempdir/defaults_dir_format",
],
},
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
# Do not use --no-sync to give test coverage for data sync.
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
defaults_parallel => {
test_key => 'defaults',
dump_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_dump', '-Fd', '-j2', "--file=$tempdir/defaults_parallel",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
restore_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_restore',
"--file=$tempdir/defaults_parallel.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
"$tempdir/defaults_parallel",
],
},
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
# Do not use --no-sync to give test coverage for data sync.
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
defaults_tar_format => {
test_key => 'defaults',
dump_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_dump', '-Ft',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
"--file=$tempdir/defaults_tar_format.tar", 'postgres',
],
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
restore_cmd => [
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'pg_restore',
'--format=tar',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
"--file=$tempdir/defaults_tar_format.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
"$tempdir/defaults_tar_format.tar",
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
exclude_dump_test_schema => {
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dump', '--no-sync',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
"--file=$tempdir/exclude_dump_test_schema.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'--exclude-schema=dump_test', 'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
exclude_test_table => {
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dump', '--no-sync',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
"--file=$tempdir/exclude_test_table.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'--exclude-table=dump_test.test_table', 'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
exclude_test_table_data => {
dump_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_dump',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
'--no-sync',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
"--file=$tempdir/exclude_test_table_data.sql",
'--exclude-table-data=dump_test.test_table',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'--no-unlogged-table-data',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
pg_dumpall_globals => {
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dumpall', '-v', "--file=$tempdir/pg_dumpall_globals.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'-g', '--no-sync',
],
},
Do not DROP default roles in pg_dumpall -c When pulling the list of roles to drop, exclude roles whose names begin with "pg_" (as we do when we are dumping the roles out to recreate them). Also add regression tests to cover pg_dumpall -c and this specific issue. Noticed by Rushabh Lathia. Patch by me.
2016-05-25 05:31:55 +02:00
pg_dumpall_globals_clean => {
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dumpall', "--file=$tempdir/pg_dumpall_globals_clean.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'-g', '-c', '--no-sync',
],
},
Correctly dump database and tablespace ACLs Dump out the appropriate GRANT/REVOKE commands for databases and tablespaces from pg_dumpall to replicate what the current state is. This was broken during the changes to buildACLCommands for 9.6+ servers for pg_init_privs.
2016-07-17 15:04:46 +02:00
pg_dumpall_dbprivs => {
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dumpall', '--no-sync',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
"--file=$tempdir/pg_dumpall_dbprivs.sql",
],
},
Add --exclude-database option to pg_dumpall This option functions similarly to pg_dump's --exclude-table option, but for database names. The option can be given once, and the argument can be a pattern including wildcard characters. Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier Discussion: https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
2019-03-01 16:47:44 +01:00
pg_dumpall_exclude => {
dump_cmd => [
'pg_dumpall', '-v', "--file=$tempdir/pg_dumpall_exclude.sql",
Avoid accidental wildcard expansion in msys shell Commit f092de05 added a test for pg_dumpall --exclude-database including the wildcard pattern '*dump*' which matches some files in the source directory. The test library on msys uses the shell which expands this and thus the program gets incorrect arguments. This doesn't happen if the pattern doesn't match any files, so here the pattern is set to '*dump_test*' which is such a pattern. Per buildfarm animal jacana.
2019-03-03 17:48:12 +01:00
'--exclude-database', '*dump_test*', '--no-sync',
Add --exclude-database option to pg_dumpall This option functions similarly to pg_dump's --exclude-table option, but for database names. The option can be given once, and the argument can be a pattern including wildcard characters. Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier Discussion: https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
2019-03-01 16:47:44 +01:00
],
},
Add --no-blobs option to pg_dump Add an option to exclude blobs when running pg_dump. By default, blobs are included but this option can be used to exclude them while keeping the rest of the dump. Commment updates and regression tests from me. Author: Guillaume Lelarge Reviewed-by: Amul Sul Discussion: https://postgr.es/m/VisenaEmail.48.49926ea6f91dceb6.15355a48249@tc7-visena
2016-11-29 17:09:35 +01:00
no_blobs => {
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dump', '--no-sync',
"--file=$tempdir/no_blobs.sql", '-B',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
no_privs => {
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dump', '--no-sync',
"--file=$tempdir/no_privs.sql", '-x',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
no_owner => {
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dump', '--no-sync',
"--file=$tempdir/no_owner.sql", '-O',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
only_dump_test_schema => {
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dump', '--no-sync',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
"--file=$tempdir/only_dump_test_schema.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'--schema=dump_test', 'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
only_dump_test_table => {
dump_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_dump',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
'--no-sync',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
"--file=$tempdir/only_dump_test_table.sql",
'--table=dump_test.test_table',
'--lock-wait-timeout=1000000',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
role => {
dump_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_dump',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
'--no-sync',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
"--file=$tempdir/role.sql",
Final pgindent + perltidy run for 9.6.
2016-08-15 19:42:51 +02:00
'--role=regress_dump_test_role',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'--schema=dump_test_second_schema',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
role_parallel => {
test_key => 'role',
dump_cmd => [
'pg_dump',
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
'--no-sync',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'--format=directory',
'--jobs=2',
"--file=$tempdir/role_parallel",
'--role=regress_dump_test_role',
'--schema=dump_test_second_schema',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
restore_cmd => [
'pg_restore', "--file=$tempdir/role_parallel.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
"$tempdir/role_parallel",
],
},
pg_dump: allow multiple rows per insert This is useful to speed up loading data in a different database engine. Authors: Surafel Temesgen and David Rowley. Lightly edited by Álvaro. Reviewed-by: Fabien Coelho Discussion: https://postgr.es/m/CALAY4q9kumSdnRBzvRJvSRf2+BH20YmSvzqOkvwpEmodD-xv6g@mail.gmail.com
2019-03-07 13:26:14 +01:00
rows_per_insert => {
dump_cmd => [
'pg_dump',
'--no-sync',
"--file=$tempdir/rows_per_insert.sql",
'-a',
'--rows-per-insert=4',
'--table=dump_test.test_table',
'--table=dump_test.test_fourth_table',
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
schema_only => {
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dump', '--format=plain',
"--file=$tempdir/schema_only.sql", '--no-sync',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'-s', 'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
section_pre_data => {
dump_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_dump', "--file=$tempdir/section_pre_data.sql",
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'--section=pre-data', '--no-sync',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
section_data => {
dump_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_dump', "--file=$tempdir/section_data.sql",
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'--section=data', '--no-sync',
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
section_post_data => {
dump_cmd => [
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
'pg_dump', "--file=$tempdir/section_post_data.sql",
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'--section=post-data', '--no-sync', 'postgres',
],
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
test_schema_plus_blobs => {
dump_cmd => [
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_dump', "--file=$tempdir/test_schema_plus_blobs.sql",
Run most pg_dump and pg_dumpall tests with --no-sync Commit 96a7128b made pg_dump and pg_dumpall sync their output by default. However, there's no great need for that in testing, and it could impose a performance penalty, so we add the --no-sync flag to most of the test cases. Michael Paquier
2017-04-11 01:53:47 +02:00
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
'--schema=dump_test', '-b', '-B', '--no-sync', 'postgres',
],
},);
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
###############################################################
# Definition of the tests to run.
#
# Each test is defined using the log message that will be used.
#
# A regexp should be defined for each test which provides the
# basis for the test. That regexp will be run against the output
# file of each of the runs which the test is to be run against
# and the success of the result will depend on if the regexp
# result matches the expected 'like' or 'unlike' case.
#
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# The runs listed as 'like' will be checked if they match the
# regexp and, if so, the test passes. All runs which are not
# listed as 'like' will be checked to ensure they don't match
# the regexp; if they do, the test will fail.
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
#
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# The below hashes provide convenience sets of runs. Individual
# runs can be excluded from a general hash by placing that run
# into the 'unlike' section.
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
#
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# For example, there is an 'exclude_test_table' run which runs a
# full pg_dump but with an exclude flag to not include the test
# table. The CREATE TABLE test which creates the test table is
# defined with %full_runs but then has 'exclude_test_table' in
# its 'unlike' list, excluding that test.
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
#
# There can then be a 'create_sql' and 'create_order' for a
# given test. The 'create_sql' commands are collected up in
# 'create_order' and then run against the database prior to any
# of the pg_dump runs happening. This is what "seeds" the
# system with objects to be dumped out.
#
# Building of this hash takes a bit of time as all of the regexps
# included in it are compiled. This greatly improves performance
# as the regexps are used for each run the test applies to.
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# Tests which target the 'dump_test' schema, specifically.
my %dump_test_schema_runs = (
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_schema => 1,
test_schema_plus_blobs => 1,);
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# Tests which are considered 'full' dumps by pg_dump, but there
# are flags used to exclude specific items (ACLs, blobs, etc).
my %full_runs = (
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
defaults => 1,
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
exclude_dump_test_schema => 1,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
exclude_test_table => 1,
exclude_test_table_data => 1,
no_blobs => 1,
no_owner => 1,
no_privs => 1,
pg_dumpall_dbprivs => 1,
Add --exclude-database option to pg_dumpall This option functions similarly to pg_dump's --exclude-table option, but for database names. The option can be given once, and the argument can be a pattern including wildcard characters. Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier Discussion: https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
2019-03-01 16:47:44 +01:00
pg_dumpall_exclude => 1,
Remove WITH OIDS support, change oid catalog column visibility. Previously tables declared WITH OIDS, including a significant fraction of the catalog tables, stored the oid column not as a normal column, but as part of the tuple header. This special column was not shown by default, which was somewhat odd, as it's often (consider e.g. pg_class.oid) one of the more important parts of a row. Neither pg_dump nor COPY included the contents of the oid column by default. The fact that the oid column was not an ordinary column necessitated a significant amount of special case code to support oid columns. That already was painful for the existing, but upcoming work aiming to make table storage pluggable, would have required expanding and duplicating that "specialness" significantly. WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0). Remove it. Removing includes: - CREATE TABLE and ALTER TABLE syntax for declaring the table to be WITH OIDS has been removed (WITH (oids[ = true]) will error out) - pg_dump does not support dumping tables declared WITH OIDS and will issue a warning when dumping one (and ignore the oid column). - restoring an pg_dump archive with pg_restore will warn when restoring a table with oid contents (and ignore the oid column) - COPY will refuse to load binary dump that includes oids. - pg_upgrade will error out when encountering tables declared WITH OIDS, they have to be altered to remove the oid column first. - Functionality to access the oid of the last inserted row (like plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed. The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false) for CREATE TABLE) is still supported. While that requires a bit of support code, it seems unnecessary to break applications / dumps that do not use oids, and are explicit about not using them. The biggest user of WITH OID columns was postgres' catalog. This commit changes all 'magic' oid columns to be columns that are normally declared and stored. To reduce unnecessary query breakage all the newly added columns are still named 'oid', even if a table's column naming scheme would indicate 'reloid' or such. This obviously requires adapting a lot code, mostly replacing oid access via HeapTupleGetOid() with access to the underlying Form_pg_*->oid column. The bootstrap process now assigns oids for all oid columns in genbki.pl that do not have an explicit value (starting at the largest oid previously used), only oids assigned later by oids will be above FirstBootstrapObjectId. As the oid column now is a normal column the special bootstrap syntax for oids has been removed. Oids are not automatically assigned during insertion anymore, all backend code explicitly assigns oids with GetNewOidWithIndex(). For the rare case that insertions into the catalog via SQL are called for the new pg_nextoid() function can be used (which only works on catalog tables). The fact that oid columns on system tables are now normal columns means that they will be included in the set of columns expanded by * (i.e. SELECT * FROM pg_class will now include the table's oid, previously it did not). It'd not technically be hard to hide oid column by default, but that'd mean confusing behavior would either have to be carried forward forever, or it'd cause breakage down the line. While it's not unlikely that further adjustments are needed, the scope/invasiveness of the patch makes it worthwhile to get merge this now. It's painful to maintain externally, too complicated to commit after the code code freeze, and a dependency of a number of other patches. Catversion bump, for obvious reasons. Author: Andres Freund, with contributions by John Naylor Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
2018-11-21 00:36:57 +01:00
schema_only => 1,);
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# This is where the actual tests are defined.
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
my %tests = (
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER DEFAULT PRIVILEGES FOR ROLE regress_dump_test_role GRANT' => {
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
create_order => 14,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'ALTER DEFAULT PRIVILEGES
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
FOR ROLE regress_dump_test_role IN SCHEMA dump_test
GRANT SELECT ON TABLES TO regress_dump_test_role;',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
\QALTER DEFAULT PRIVILEGES \E
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
\QFOR ROLE regress_dump_test_role IN SCHEMA dump_test \E
\QGRANT SELECT ON TABLES TO regress_dump_test_role;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_privs => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER DEFAULT PRIVILEGES FOR ROLE regress_dump_test_role REVOKE' => {
create_order => 55,
create_sql => 'ALTER DEFAULT PRIVILEGES
FOR ROLE regress_dump_test_role
REVOKE EXECUTE ON FUNCTIONS FROM PUBLIC;',
regexp => qr/^
\QALTER DEFAULT PRIVILEGES \E
\QFOR ROLE regress_dump_test_role \E
\QREVOKE ALL ON FUNCTIONS FROM PUBLIC;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_post_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_privs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER DEFAULT PRIVILEGES FOR ROLE regress_dump_test_role REVOKE SELECT'
=> {
create_order => 56,
create_sql => 'ALTER DEFAULT PRIVILEGES
FOR ROLE regress_dump_test_role
REVOKE SELECT ON TABLES FROM regress_dump_test_role;',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\QALTER DEFAULT PRIVILEGES \E
\QFOR ROLE regress_dump_test_role \E
\QREVOKE ALL ON TABLES FROM regress_dump_test_role;\E\n
\QALTER DEFAULT PRIVILEGES \E
\QFOR ROLE regress_dump_test_role \E
\QGRANT INSERT,REFERENCES,DELETE,TRIGGER,TRUNCATE,UPDATE ON TABLES TO regress_dump_test_role;\E
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_post_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_privs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER ROLE regress_dump_test_role' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\QALTER ROLE regress_dump_test_role WITH \E
\QNOSUPERUSER INHERIT NOCREATEROLE NOCREATEDB NOLOGIN \E
\QNOREPLICATION NOBYPASSRLS;\E
/xm,
like => {
pg_dumpall_dbprivs => 1,
pg_dumpall_globals => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
pg_dumpall_globals_clean => 1,
Add --exclude-database option to pg_dumpall This option functions similarly to pg_dump's --exclude-table option, but for database names. The option can be given once, and the argument can be a pattern including wildcard characters. Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier Discussion: https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
2019-03-01 16:47:44 +01:00
pg_dumpall_exclude => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER COLLATION test0 OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^\QALTER COLLATION public.test0 OWNER TO \E.+;/m,
pg_dump: Skip COLLATION-related regression tests Not every platform supports non-default collations, as pointed out by the buildfarm, so skip collation-related regression tests in pg_dump when they aren't supported.
2017-03-19 21:56:14 +01:00
collation => 1,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { %dump_test_schema_runs, no_owner => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER FOREIGN DATA WRAPPER dummy OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^ALTER FOREIGN DATA WRAPPER dummy OWNER TO .+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_owner => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER SERVER s1 OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^ALTER SERVER s1 OWNER TO .+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_owner => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER FUNCTION dump_test.pltestlang_call_handler() OWNER TO' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\QALTER FUNCTION dump_test.pltestlang_call_handler() \E
\QOWNER TO \E
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
.+;/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Correctly handle owned sequences with extensions With the refactoring of pg_dump to handle components, getOwnedSeqs needs to be a bit more intelligent regarding which components to dump when. Specifically, we can't simply use the owning table's components as the set of components to dump as the table might only be including certain components while all components of the sequence should be dumped, for example, when the table is a member of an extension while the sequence is not. Handle this by combining the set of components to be dumped for the sequence explicitly and those to be dumped for the table when setting the components to be dumped for the sequence. Also add a number of regression tests around this to, hopefully, catch any future changes which break the expected behavior. Discovered by: Philippe BEAUDOIN Reviewed by: Michael Paquier
2016-07-31 16:57:15 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_owner => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER OPERATOR FAMILY dump_test.op_family OWNER TO' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\QALTER OPERATOR FAMILY dump_test.op_family USING btree \E
\QOWNER TO \E
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
.+;/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_owner => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER OPERATOR FAMILY dump_test.op_family USING btree' => {
create_order => 75,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
'ALTER OPERATOR FAMILY dump_test.op_family USING btree ADD
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
OPERATOR 1 <(bigint,int4),
OPERATOR 2 <=(bigint,int4),
OPERATOR 3 =(bigint,int4),
OPERATOR 4 >=(bigint,int4),
OPERATOR 5 >(bigint,int4),
FUNCTION 1 (int4, int4) btint4cmp(int4,int4),
FUNCTION 2 (int4, int4) btint4sortsupport(internal);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER OPERATOR FAMILY dump_test.op_family USING btree ADD\E\n\s+
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\QOPERATOR 1 <(bigint,integer) ,\E\n\s+
\QOPERATOR 2 <=(bigint,integer) ,\E\n\s+
\QOPERATOR 3 =(bigint,integer) ,\E\n\s+
\QOPERATOR 4 >=(bigint,integer) ,\E\n\s+
\QOPERATOR 5 >(bigint,integer) ,\E\n\s+
\QFUNCTION 1 (integer, integer) btint4cmp(integer,integer) ,\E\n\s+
\QFUNCTION 2 (integer, integer) btint4sortsupport(internal);\E
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER OPERATOR CLASS dump_test.op_class OWNER TO' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\QALTER OPERATOR CLASS dump_test.op_class USING btree \E
\QOWNER TO \E
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
.+;/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_owner => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER PUBLICATION pub1 OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^ALTER PUBLICATION pub1 OWNER TO .+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_post_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_owner => 1, },
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER LARGE OBJECT ... OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^ALTER LARGE OBJECT \d+ OWNER TO .+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
column_inserts => 1,
data_only => 1,
section_pre_data => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
test_schema_plus_blobs => 1,
},
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
no_blobs => 1,
no_owner => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER PROCEDURAL LANGUAGE pltestlang OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^ALTER PROCEDURAL LANGUAGE pltestlang OWNER TO .+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_owner => 1, },
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER SCHEMA dump_test OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^ALTER SCHEMA dump_test OWNER TO .+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_owner => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER SCHEMA dump_test_second_schema OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^ALTER SCHEMA dump_test_second_schema OWNER TO .+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
role => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
unlike => { no_owner => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER SEQUENCE test_table_col1_seq' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER SEQUENCE dump_test.test_table_col1_seq OWNED BY dump_test.test_table.col1;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE ONLY test_table ADD CONSTRAINT ... PRIMARY KEY' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE ONLY dump_test.test_table\E \n^\s+
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\QADD CONSTRAINT test_table_pkey PRIMARY KEY (col1);\E
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
Dump foreign keys on partitioned tables The patch that ended up as commit 3de241dba86f ("Foreign keys on partitioned tables") lacked pg_dump tests, so the pg_dump code that was there to support it inadvertently stopped working when in later development I modified the backend code not to emit pg_trigger rows for the partitioned table itself. Bug analysis and code fix is by Michaël. I (Álvaro) added the test. Reported-by: amul sul <sulamul@gmail.com> Co-authored-by: Michaël Paquier <michael@paquier.xyz> Co-authored-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Discussion: https://postgr.es/m/CAAJ_b94n=UsNVhgs97vCaWEZAMe-tGDRVuZ73oePQH=eaJKGSA@mail.gmail.com
2018-07-13 19:13:26 +02:00
'ALTER TABLE (partitioned) ADD CONSTRAINT ... FOREIGN KEY' => {
create_order => 4,
create_sql => 'CREATE TABLE dump_test.test_table_fk (
col1 int references dump_test.test_table)
PARTITION BY RANGE (col1);
CREATE TABLE dump_test.test_table_fk_1
PARTITION OF dump_test.test_table_fk
FOR VALUES FROM (0) TO (10);',
regexp => qr/
\QADD CONSTRAINT test_table_fk_col1_fkey FOREIGN KEY (col1) REFERENCES dump_test.test_table\E
/xm,
like => {
%full_runs, %dump_test_schema_runs, section_post_data => 1,
},
unlike => {
exclude_dump_test_schema => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
create_order => 93,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'ALTER TABLE dump_test.test_table ALTER COLUMN col1 SET STATISTICS 90;',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE ONLY dump_test.test_table ALTER COLUMN col1 SET STATISTICS 90;\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE ONLY test_table ALTER COLUMN col2 SET STORAGE' => {
create_order => 94,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'ALTER TABLE dump_test.test_table ALTER COLUMN col2 SET STORAGE EXTERNAL;',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE ONLY dump_test.test_table ALTER COLUMN col2 SET STORAGE EXTERNAL;\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE ONLY test_table ALTER COLUMN col3 SET STORAGE' => {
create_order => 95,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'ALTER TABLE dump_test.test_table ALTER COLUMN col3 SET STORAGE MAIN;',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE ONLY dump_test.test_table ALTER COLUMN col3 SET STORAGE MAIN;\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE ONLY test_table ALTER COLUMN col4 SET n_distinct' => {
create_order => 95,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'ALTER TABLE dump_test.test_table ALTER COLUMN col4 SET (n_distinct = 10);',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE ONLY dump_test.test_table ALTER COLUMN col4 SET (n_distinct=10);\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'ALTER TABLE ONLY dump_test.measurement ATTACH PARTITION measurement_y2006m2'
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
=> {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE ONLY dump_test.measurement ATTACH PARTITION dump_test_second_schema.measurement_y2006m2 \E
Change the way pg_dump retrieves partitioning info This gets rid of the code that issued separate queries to retrieve the partitioning parent-child relationship, parent partition key, and child partition bound information. With this patch, the information is retrieved instead using the queries issued from getTables() and getInherits(), which is both more efficient than the previous approach and doesn't require any new code. Since the partitioning parent-child relationship is now retrieved with the same old code that handles inheritance, partition attributes receive a proper flagInhAttrs() treatment (that it didn't receive before), which is needed so that the inherited NOT NULL constraints are not emitted if we already emitted it for the parent. Also, fix a bug in pg_dump's --binary-upgrade code, which caused pg_dump to emit invalid command to attach a partition to its parent. Author: Amit Langote, with some additional changes by me.
2017-05-05 04:17:52 +02:00
\QFOR VALUES FROM ('2006-02-01') TO ('2006-03-01');\E\n
/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { binary_upgrade => 1, },
},
Change the way pg_dump retrieves partitioning info This gets rid of the code that issued separate queries to retrieve the partitioning parent-child relationship, parent partition key, and child partition bound information. With this patch, the information is retrieved instead using the queries issued from getTables() and getInherits(), which is both more efficient than the previous approach and doesn't require any new code. Since the partitioning parent-child relationship is now retrieved with the same old code that handles inheritance, partition attributes receive a proper flagInhAttrs() treatment (that it didn't receive before), which is needed so that the inherited NOT NULL constraints are not emitted if we already emitted it for the parent. Also, fix a bug in pg_dump's --binary-upgrade code, which caused pg_dump to emit invalid command to attach a partition to its parent. Author: Amit Langote, with some additional changes by me.
2017-05-05 04:17:52 +02:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE test_table CLUSTER ON test_table_pkey' => {
create_order => 96,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
'ALTER TABLE dump_test.test_table CLUSTER ON test_table_pkey',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE dump_test.test_table CLUSTER ON test_table_pkey;\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE test_table DISABLE TRIGGER ALL' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\QSET SESSION AUTHORIZATION 'test_superuser';\E\n\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE dump_test.test_table DISABLE TRIGGER ALL;\E\n\n
\QCOPY dump_test.test_table (col1, col2, col3, col4) FROM stdin;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n(?:\d\t\\N\t\\N\t\\N\n){9}\\\.\n\n\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE dump_test.test_table ENABLE TRIGGER ALL;\E/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { data_only => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER FOREIGN TABLE foreign_table ALTER COLUMN c1 OPTIONS' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER FOREIGN TABLE dump_test.foreign_table ALTER COLUMN c1 OPTIONS (\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\Qcolumn_name 'col1'\E\n
\Q);\E\n
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE test_table OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^\QALTER TABLE dump_test.test_table OWNER TO \E.+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
exclude_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_owner => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE test_table ENABLE ROW LEVEL SECURITY' => {
create_order => 23,
create_sql => 'ALTER TABLE dump_test.test_table
ENABLE ROW LEVEL SECURITY;',
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QALTER TABLE dump_test.test_table ENABLE ROW LEVEL SECURITY;\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE test_second_table OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^\QALTER TABLE dump_test.test_second_table OWNER TO \E.+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_owner => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE measurement OWNER TO' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^\QALTER TABLE dump_test.measurement OWNER TO \E.+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_owner => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TABLE measurement_y2006m2 OWNER TO' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp =>
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
qr/^\QALTER TABLE dump_test_second_schema.measurement_y2006m2 OWNER TO \E.+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
role => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
unlike => { no_owner => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER FOREIGN TABLE foreign_table OWNER TO' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp =>
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
qr/^\QALTER FOREIGN TABLE dump_test.foreign_table OWNER TO \E.+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_owner => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TEXT SEARCH CONFIGURATION alt_ts_conf1 OWNER TO' => {
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
qr/^\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1 OWNER TO \E.+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_owner => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TEXT SEARCH DICTIONARY alt_ts_dict1 OWNER TO' => {
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
qr/^\QALTER TEXT SEARCH DICTIONARY dump_test.alt_ts_dict1 OWNER TO \E.+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
only_dump_test_table => 1,
no_owner => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
role => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'BLOB create (using lo_from_bytea)' => {
create_order => 50,
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'SELECT pg_catalog.lo_from_bytea(0, \'\\x310a320a330a340a350a360a370a380a390a\');',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^SELECT pg_catalog\.lo_create\('\d+'\);/m,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
column_inserts => 1,
data_only => 1,
section_pre_data => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
test_schema_plus_blobs => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
schema_only => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_blobs => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'BLOB load (using lo_from_bytea)' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\QSELECT pg_catalog.lo_open\E \('\d+',\ \d+\);\n
\QSELECT pg_catalog.lowrite(0, \E
\Q'\x310a320a330a340a350a360a370a380a390a');\E\n
\QSELECT pg_catalog.lo_close(0);\E
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
column_inserts => 1,
data_only => 1,
section_data => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
test_schema_plus_blobs => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
binary_upgrade => 1,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
no_blobs => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON DATABASE postgres' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^COMMENT ON DATABASE postgres IS .+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
Move handling of database properties from pg_dumpall into pg_dump. This patch rearranges the division of labor between pg_dump and pg_dumpall so that pg_dump itself handles all properties attached to a single database. Notably, a database's ACL (GRANT/REVOKE status) and local GUC settings established by ALTER DATABASE SET and ALTER ROLE IN DATABASE SET can be dumped and restored by pg_dump. This is a long-requested improvement. "pg_dumpall -g" will now produce only role- and tablespace-related output, nothing about individual databases. The total output of a regular pg_dumpall run remains the same. pg_dump (or pg_restore) will restore database-level properties only when creating the target database with --create. This applies not only to ACLs and GUCs but to the other database properties it already handled, that is database comments and security labels. This is more consistent and useful, but does represent an incompatibility in the behavior seen without --create. (This change makes the proposed patch to have pg_dump use "COMMENT ON DATABASE CURRENT_DATABASE" unnecessary, since there is no case where the command is issued that we won't know the true name of the database. We might still want that patch as a feature in its own right, but pg_dump no longer needs it.) pg_dumpall with --clean will now drop and recreate the "postgres" and "template1" databases in the target cluster, allowing their locale and encoding settings to be changed if necessary, and providing a cleaner way to set nondefault tablespaces for them than we had before. This means that such a script must now always be started in the "postgres" database; the order of drops and reconnects will not work otherwise. Without --clean, the script will not adjust any database-level properties of those two databases (including their comments, ACLs, and security labels, which it formerly would try to set). Another minor incompatibility is that the CREATE DATABASE commands in a pg_dumpall script will now always specify locale and encoding settings. Formerly those would be omitted if they matched the cluster's default. While that behavior had some usefulness in some migration scenarios, it also posed a significant hazard of unwanted locale/encoding changes. To migrate to another locale/encoding, it's now necessary to use pg_dump without --create to restore into a database with the desired settings. Commit 4bd371f6f's hack to emit "SET default_transaction_read_only = off" is gone: we now dodge that problem by the expedient of not issuing ALTER DATABASE SET commands until after reconnecting to the target database. Therefore, such settings won't apply during the restore session. In passing, improve some shaky grammar in the docs, and add a note pointing out that pg_dumpall's output can't be expected to load without any errors. (Someday we might want to fix that, but this is not that patch.) Haribabu Kommi, reviewed at various times by Andreas Karlsson, Vaishnavi Prabakaran, and Robert Haas; further hacking by me. Discussion: https://postgr.es/m/CAJrrPGcUurV0eWTeXODwsOYFN=Ekq36t1s0YnFYUNzsmRfdAyA@mail.gmail.com
2018-01-22 20:09:09 +01:00
# Should appear in the same tests as "CREATE DATABASE postgres"
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { createdb => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON EXTENSION plpgsql' => {
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
regexp => qr/^COMMENT ON EXTENSION plpgsql IS .+;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
Improve pg_dump's handling of "special" built-in objects. We had some pretty ad-hoc handling of the public schema and the plpgsql extension, which are both presumed to exist in template0 but might be modified or deleted in the database being dumped. Up to now, by default pg_dump would emit a CREATE EXTENSION IF NOT EXISTS command as well as a COMMENT command for plpgsql. The usefulness of the former is questionable, and the latter caused annoying errors in non-superuser dump/restore scenarios. Let's instead install a rule that built-in extensions (identified by having low-numbered OIDs) are not to be dumped. We were doing it that way already in binary-upgrade mode, so this just makes regular mode behave the same. It remains true that if someone has installed a non-default ACL on the plpgsql language, that will get dumped thanks to the pg_init_privs mechanism. This is more consistent with the handling of built-in objects of other kinds. Also, change the very ad-hoc mechanism that was used to avoid dumping creation and comment commands for the public schema. Instead of hardwiring a test in _printTocEntry(), make use of the DUMP_COMPONENT_ infrastructure to mark that schema up-front about what we want to do with it. This has the visible effect that the public schema won't be mentioned in the output at all, except for updating its ACL if it has a non-default ACL. Previously, while it was normally not mentioned, --clean mode would drop and recreate it, again causing headaches for non-superuser usage. This change likewise makes the public schema less special and more like other built-in objects. If plpgsql, or the public schema, has been removed entirely in the source DB, that situation won't be reproduced in the destination ... but that was true before. Discussion: https://postgr.es/m/29048.1516812451@sss.pgh.pa.us
2018-01-25 19:54:42 +01:00
# this shouldn't ever get emitted anymore
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON TABLE dump_test.test_table' => {
create_order => 36,
create_sql => 'COMMENT ON TABLE dump_test.test_table
IS \'comment on table\';',
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON TABLE dump_test.test_table IS 'comment on table';\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'COMMENT ON COLUMN dump_test.test_table.col1' => {
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
create_order => 36,
create_sql => 'COMMENT ON COLUMN dump_test.test_table.col1
IS \'comment on column\';',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOMMENT ON COLUMN dump_test.test_table.col1 IS 'comment on column';\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON COLUMN dump_test.composite.f1' => {
create_order => 44,
create_sql => 'COMMENT ON COLUMN dump_test.composite.f1
IS \'comment on column of type\';',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOMMENT ON COLUMN dump_test.composite.f1 IS 'comment on column of type';\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON COLUMN dump_test.test_second_table.col1' => {
create_order => 63,
create_sql => 'COMMENT ON COLUMN dump_test.test_second_table.col1
IS \'comment on column col1\';',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOMMENT ON COLUMN dump_test.test_second_table.col1 IS 'comment on column col1';\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON COLUMN dump_test.test_second_table.col2' => {
create_order => 64,
create_sql => 'COMMENT ON COLUMN dump_test.test_second_table.col2
IS \'comment on column col2\';',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOMMENT ON COLUMN dump_test.test_second_table.col2 IS 'comment on column col2';\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON CONVERSION dump_test.test_conversion' => {
create_order => 79,
create_sql => 'COMMENT ON CONVERSION dump_test.test_conversion
IS \'comment on test conversion\';',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON CONVERSION dump_test.test_conversion IS 'comment on test conversion';\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON COLLATION test0' => {
create_order => 77,
create_sql => 'COMMENT ON COLLATION test0
IS \'comment on test0 collation\';',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON COLLATION public.test0 IS 'comment on test0 collation';\E/m,
pg_dump: Skip COLLATION-related regression tests Not every platform supports non-default collations, as pointed out by the buildfarm, so skip collation-related regression tests in pg_dump when they aren't supported.
2017-03-19 21:56:14 +01:00
collation => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_pre_data => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON LARGE OBJECT ...' => {
create_order => 65,
create_sql => 'DO $$
DECLARE myoid oid;
BEGIN
SELECT loid FROM pg_largeobject INTO myoid;
EXECUTE \'COMMENT ON LARGE OBJECT \' || myoid || \' IS \'\'comment on large object\'\';\';
END;
$$;',
regexp => qr/^
\QCOMMENT ON LARGE OBJECT \E[0-9]+\Q IS 'comment on large object';\E
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
column_inserts => 1,
data_only => 1,
section_pre_data => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
test_schema_plus_blobs => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
no_blobs => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
pg_dump: Dump comments and security labels for publication and subscriptions
2017-04-14 04:32:03 +02:00
'COMMENT ON PUBLICATION pub1' => {
create_order => 55,
create_sql => 'COMMENT ON PUBLICATION pub1
IS \'comment on publication\';',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
qr/^COMMENT ON PUBLICATION pub1 IS 'comment on publication';/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_post_data => 1, },
},
pg_dump: Dump comments and security labels for publication and subscriptions
2017-04-14 04:32:03 +02:00
'COMMENT ON SUBSCRIPTION sub1' => {
create_order => 55,
create_sql => 'COMMENT ON SUBSCRIPTION sub1
IS \'comment on subscription\';',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
qr/^COMMENT ON SUBSCRIPTION sub1 IS 'comment on subscription';/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_post_data => 1, },
},
pg_dump: Dump comments and security labels for publication and subscriptions
2017-04-14 04:32:03 +02:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1' => {
create_order => 84,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
'COMMENT ON TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
IS \'comment on text search configuration\';',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1 IS 'comment on text search configuration';\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON TEXT SEARCH DICTIONARY dump_test.alt_ts_dict1' => {
create_order => 84,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
'COMMENT ON TEXT SEARCH DICTIONARY dump_test.alt_ts_dict1
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
IS \'comment on text search dictionary\';',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON TEXT SEARCH DICTIONARY dump_test.alt_ts_dict1 IS 'comment on text search dictionary';\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON TEXT SEARCH PARSER dump_test.alt_ts_prs1' => {
create_order => 84,
create_sql => 'COMMENT ON TEXT SEARCH PARSER dump_test.alt_ts_prs1
IS \'comment on text search parser\';',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON TEXT SEARCH PARSER dump_test.alt_ts_prs1 IS 'comment on text search parser';\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON TEXT SEARCH TEMPLATE dump_test.alt_ts_temp1' => {
create_order => 84,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql => 'COMMENT ON TEXT SEARCH TEMPLATE dump_test.alt_ts_temp1
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
IS \'comment on text search template\';',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON TEXT SEARCH TEMPLATE dump_test.alt_ts_temp1 IS 'comment on text search template';\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON TYPE dump_test.planets - ENUM' => {
create_order => 68,
create_sql => 'COMMENT ON TYPE dump_test.planets
IS \'comment on enum type\';',
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON TYPE dump_test.planets IS 'comment on enum type';\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON TYPE dump_test.textrange - RANGE' => {
create_order => 69,
create_sql => 'COMMENT ON TYPE dump_test.textrange
IS \'comment on range type\';',
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON TYPE dump_test.textrange IS 'comment on range type';\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON TYPE dump_test.int42 - Regular' => {
create_order => 70,
create_sql => 'COMMENT ON TYPE dump_test.int42
IS \'comment on regular type\';',
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON TYPE dump_test.int42 IS 'comment on regular type';\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COMMENT ON TYPE dump_test.undefined - Undefined' => {
create_order => 71,
create_sql => 'COMMENT ON TYPE dump_test.undefined
IS \'comment on undefined type\';',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QCOMMENT ON TYPE dump_test.undefined IS 'comment on undefined type';\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'COPY test_table' => {
create_order => 4,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'INSERT INTO dump_test.test_table (col1) '
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
. 'SELECT generate_series FROM generate_series(1,9);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOPY dump_test.test_table (col1, col2, col3, col4) FROM stdin;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n(?:\d\t\\N\t\\N\t\\N\n){9}\\\.\n
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
data_only => 1,
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_data => 1,
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
exclude_dump_test_schema => 1,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
exclude_test_table => 1,
exclude_test_table_data => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'COPY fk_reference_test_table' => {
create_order => 22,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'INSERT INTO dump_test.fk_reference_test_table (col1) '
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
. 'SELECT generate_series FROM generate_series(1,5);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOPY dump_test.fk_reference_test_table (col1) FROM stdin;\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n(?:\d\n){5}\\\.\n
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
data_only => 1,
exclude_test_table => 1,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
exclude_test_table_data => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_data => 1,
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# In a data-only dump, we try to actually order according to FKs,
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
# so this check is just making sure that the referring table comes after
# the referred-to table.
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'COPY fk_reference_test_table second' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOPY dump_test.test_table (col1, col2, col3, col4) FROM stdin;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n(?:\d\t\\N\t\\N\t\\N\n){9}\\\.\n.*
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOPY dump_test.fk_reference_test_table (col1) FROM stdin;\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n(?:\d\n){5}\\\.\n
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xms,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { data_only => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'COPY test_second_table' => {
create_order => 7,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'INSERT INTO dump_test.test_second_table (col1, col2) '
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
. 'SELECT generate_series, generate_series::text '
. 'FROM generate_series(1,9);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOPY dump_test.test_second_table (col1, col2) FROM stdin;\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n(?:\d\t\d\n){9}\\\.\n
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
data_only => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_data => 1,
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COPY test_fourth_table' => {
create_order => 7,
create_sql =>
pg_dump: allow multiple rows per insert This is useful to speed up loading data in a different database engine. Authors: Surafel Temesgen and David Rowley. Lightly edited by Álvaro. Reviewed-by: Fabien Coelho Discussion: https://postgr.es/m/CALAY4q9kumSdnRBzvRJvSRf2+BH20YmSvzqOkvwpEmodD-xv6g@mail.gmail.com
2019-03-07 13:26:14 +01:00
'INSERT INTO dump_test.test_fourth_table DEFAULT VALUES;'
. 'INSERT INTO dump_test.test_fourth_table DEFAULT VALUES;',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOPY dump_test.test_fourth_table FROM stdin;\E
pg_dump: allow multiple rows per insert This is useful to speed up loading data in a different database engine. Authors: Surafel Temesgen and David Rowley. Lightly edited by Álvaro. Reviewed-by: Fabien Coelho Discussion: https://postgr.es/m/CALAY4q9kumSdnRBzvRJvSRf2+BH20YmSvzqOkvwpEmodD-xv6g@mail.gmail.com
2019-03-07 13:26:14 +01:00
\n\n\n\\\.\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
data_only => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'COPY test_fifth_table' => {
create_order => 54,
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'INSERT INTO dump_test.test_fifth_table VALUES (NULL, true, false, \'11001\'::bit(5), \'NaN\');',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOPY dump_test.test_fifth_table (col1, col2, col3, col4, col5) FROM stdin;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n\\N\tt\tf\t11001\tNaN\n\\\.\n
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
data_only => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_data => 1,
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
'COPY test_table_identity' => {
create_order => 54,
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'INSERT INTO dump_test.test_table_identity (col2) VALUES (\'test\');',
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCOPY dump_test.test_table_identity (col1, col2) FROM stdin;\E
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
\n1\ttest\n\\\.\n
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
data_only => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_data => 1,
},
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'INSERT INTO test_table' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
(?:INSERT\ INTO\ dump_test\.test_table\ \(col1,\ col2,\ col3,\ col4\)\ VALUES\ \(\d,\ NULL,\ NULL,\ NULL\);\n){9}
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { column_inserts => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
pg_dump: allow multiple rows per insert This is useful to speed up loading data in a different database engine. Authors: Surafel Temesgen and David Rowley. Lightly edited by Álvaro. Reviewed-by: Fabien Coelho Discussion: https://postgr.es/m/CALAY4q9kumSdnRBzvRJvSRf2+BH20YmSvzqOkvwpEmodD-xv6g@mail.gmail.com
2019-03-07 13:26:14 +01:00
'test_table with 4-row INSERTs' => {
regexp => qr/^
(?:
INSERT\ INTO\ dump_test\.test_table\ VALUES\n
(?:\t\(\d,\ NULL,\ NULL,\ NULL\),\n){3}
\t\(\d,\ NULL,\ NULL,\ NULL\);\n
){2}
INSERT\ INTO\ dump_test\.test_table\ VALUES\n
\t\(\d,\ NULL,\ NULL,\ NULL\);
/xm,
like => { rows_per_insert => 1, },
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'INSERT INTO test_second_table' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
(?:INSERT\ INTO\ dump_test\.test_second_table\ \(col1,\ col2\)
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
\ VALUES\ \(\d,\ '\d'\);\n){9}/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { column_inserts => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'INSERT INTO test_fourth_table' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp =>
pg_dump: allow multiple rows per insert This is useful to speed up loading data in a different database engine. Authors: Surafel Temesgen and David Rowley. Lightly edited by Álvaro. Reviewed-by: Fabien Coelho Discussion: https://postgr.es/m/CALAY4q9kumSdnRBzvRJvSRf2+BH20YmSvzqOkvwpEmodD-xv6g@mail.gmail.com
2019-03-07 13:26:14 +01:00
qr/^(?:INSERT INTO dump_test\.test_fourth_table DEFAULT VALUES;\n){2}/m,
like => { column_inserts => 1, rows_per_insert => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'INSERT INTO test_fifth_table' => {
regexp =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
qr/^\QINSERT INTO dump_test.test_fifth_table (col1, col2, col3, col4, col5) VALUES (NULL, true, false, B'11001', 'NaN');\E/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { column_inserts => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
'INSERT INTO test_table_identity' => {
regexp =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
qr/^\QINSERT INTO dump_test.test_table_identity (col1, col2) OVERRIDING SYSTEM VALUE VALUES (1, 'test');\E/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { column_inserts => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
'CREATE ROLE regress_dump_test_role' => {
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
create_order => 1,
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
create_sql => 'CREATE ROLE regress_dump_test_role;',
regexp => qr/^CREATE ROLE regress_dump_test_role;/m,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
like => {
Correctly dump database and tablespace ACLs Dump out the appropriate GRANT/REVOKE commands for databases and tablespaces from pg_dumpall to replicate what the current state is. This was broken during the changes to buildACLCommands for 9.6+ servers for pg_init_privs.
2016-07-17 15:04:46 +02:00
pg_dumpall_dbprivs => 1,
Add --exclude-database option to pg_dumpall This option functions similarly to pg_dump's --exclude-table option, but for database names. The option can be given once, and the argument can be a pattern including wildcard characters. Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier Discussion: https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
2019-03-01 16:47:44 +01:00
pg_dumpall_exclude => 1,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
pg_dumpall_globals => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
pg_dumpall_globals_clean => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE ACCESS METHOD gist2' => {
create_order => 52,
create_sql =>
'CREATE ACCESS METHOD gist2 TYPE INDEX HANDLER gisthandler;',
regexp =>
qr/CREATE ACCESS METHOD gist2 TYPE INDEX HANDLER gisthandler;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_pre_data => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql => 'CREATE COLLATION test0 FROM "C";',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE COLLATION public.test0 (provider = libc, locale = 'C');\E/xm,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
collation => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_pre_data => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Fix dumping of casts and transforms using built-in functions In pg_dump.c dumpCast() and dumpTransform(), we would happily ignore the cast or transform if it happened to use a built-in function because we weren't including the information about built-in functions when querying pg_proc from getFuncs(). Modify the query in getFuncs() to also gather information about functions which are used by user-defined casts and transforms (where "user-defined" means "has an OID >= FirstNormalObjectId"). This also adds to the TAP regression tests for 9.6 and master to cover these types of objects. Back-patch all the way for casts, back to 9.5 for transforms. Discussion: https://www.postgresql.org/message-id/flat/20160504183952.GE10850%40tamriel.snowman.net
2016-12-21 19:47:06 +01:00
'CREATE CAST FOR timestamptz' => {
create_order => 51,
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'CREATE CAST (timestamptz AS interval) WITH FUNCTION age(timestamptz) AS ASSIGNMENT;',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
regexp =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
qr/CREATE CAST \(timestamp with time zone AS interval\) WITH FUNCTION pg_catalog\.age\(timestamp with time zone\) AS ASSIGNMENT;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_pre_data => 1, },
},
Fix dumping of casts and transforms using built-in functions In pg_dump.c dumpCast() and dumpTransform(), we would happily ignore the cast or transform if it happened to use a built-in function because we weren't including the information about built-in functions when querying pg_proc from getFuncs(). Modify the query in getFuncs() to also gather information about functions which are used by user-defined casts and transforms (where "user-defined" means "has an OID >= FirstNormalObjectId"). This also adds to the TAP regression tests for 9.6 and master to cover these types of objects. Back-patch all the way for casts, back to 9.5 for transforms. Discussion: https://www.postgresql.org/message-id/flat/20160504183952.GE10850%40tamriel.snowman.net
2016-12-21 19:47:06 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE DATABASE postgres' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\QCREATE DATABASE postgres WITH TEMPLATE = template0 \E
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
.+;/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { createdb => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Correctly dump database and tablespace ACLs Dump out the appropriate GRANT/REVOKE commands for databases and tablespaces from pg_dumpall to replicate what the current state is. This was broken during the changes to buildACLCommands for 9.6+ servers for pg_init_privs.
2016-07-17 15:04:46 +02:00
'CREATE DATABASE dump_test' => {
create_order => 47,
Final pgindent + perltidy run for 9.6.
2016-08-15 19:42:51 +02:00
create_sql => 'CREATE DATABASE dump_test;',
regexp => qr/^
Correctly dump database and tablespace ACLs Dump out the appropriate GRANT/REVOKE commands for databases and tablespaces from pg_dumpall to replicate what the current state is. This was broken during the changes to buildACLCommands for 9.6+ servers for pg_init_privs.
2016-07-17 15:04:46 +02:00
\QCREATE DATABASE dump_test WITH TEMPLATE = template0 \E
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
.+;/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { pg_dumpall_dbprivs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE EXTENSION ... plpgsql' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\QCREATE EXTENSION IF NOT EXISTS plpgsql WITH SCHEMA pg_catalog;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
Improve pg_dump's handling of "special" built-in objects. We had some pretty ad-hoc handling of the public schema and the plpgsql extension, which are both presumed to exist in template0 but might be modified or deleted in the database being dumped. Up to now, by default pg_dump would emit a CREATE EXTENSION IF NOT EXISTS command as well as a COMMENT command for plpgsql. The usefulness of the former is questionable, and the latter caused annoying errors in non-superuser dump/restore scenarios. Let's instead install a rule that built-in extensions (identified by having low-numbered OIDs) are not to be dumped. We were doing it that way already in binary-upgrade mode, so this just makes regular mode behave the same. It remains true that if someone has installed a non-default ACL on the plpgsql language, that will get dumped thanks to the pg_init_privs mechanism. This is more consistent with the handling of built-in objects of other kinds. Also, change the very ad-hoc mechanism that was used to avoid dumping creation and comment commands for the public schema. Instead of hardwiring a test in _printTocEntry(), make use of the DUMP_COMPONENT_ infrastructure to mark that schema up-front about what we want to do with it. This has the visible effect that the public schema won't be mentioned in the output at all, except for updating its ACL if it has a non-default ACL. Previously, while it was normally not mentioned, --clean mode would drop and recreate it, again causing headaches for non-superuser usage. This change likewise makes the public schema less special and more like other built-in objects. If plpgsql, or the public schema, has been removed entirely in the source DB, that situation won't be reproduced in the destination ... but that was true before. Discussion: https://postgr.es/m/29048.1516812451@sss.pgh.pa.us
2018-01-25 19:54:42 +01:00
# this shouldn't ever get emitted anymore
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE AGGREGATE dump_test.newavg' => {
create_order => 25,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE AGGREGATE dump_test.newavg (
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
sfunc = int4_avg_accum,
basetype = int4,
stype = _int8,
finalfunc = int8_avg,
Improve spelling of new FINALFUNC_MODIFY aggregate attribute. I'd used SHARABLE as a value originally, but Peter Eisentraut points out that dictionaries agree that SHAREABLE is the preferred spelling. Run around and change that before it's too late. Discussion: https://postgr.es/m/d2e1afd4-659c-50d6-1b20-7cfd3675e909@2ndquadrant.com
2018-05-21 17:41:42 +02:00
finalfunc_modify = shareable,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
initcond1 = \'{0,0}\'
);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE AGGREGATE dump_test.newavg(integer) (\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QSFUNC = int4_avg_accum,\E
\n\s+\QSTYPE = bigint[],\E
\n\s+\QINITCOND = '{0,0}',\E
Explicitly track whether aggregate final functions modify transition state. Up to now, there's been hard-wired assumptions that normal aggregates' final functions never modify their transition states, while ordered-set aggregates' final functions always do. This has always been a bit limiting, and in particular it's getting in the way of improving the built-in ordered-set aggregates to allow merging of transition states. Therefore, let's introduce catalog and CREATE AGGREGATE infrastructure that lets the finalfn's behavior be declared explicitly. There are now three possibilities for the finalfn behavior: it's purely read-only, it trashes the transition state irrecoverably, or it changes the state in such a way that no more transfn calls are possible but the state can still be passed to other, compatible finalfns. There are no examples of this third case today, but we'll shortly make the built-in OSAs act like that. This change allows user-defined aggregates to explicitly disclaim support for use as window functions, and/or to prevent transition state merging, if their implementations cannot handle that. While it was previously possible to handle the window case with a run-time error check, there was not any way to prevent transition state merging, which in retrospect is something commit 804163bc2 should have provided for. But better late than never. In passing, split out pg_aggregate.c's extern function declarations into a new header file pg_aggregate_fn.h, similarly to what we've done for some other catalog headers, so that pg_aggregate.h itself can be safe for frontend files to include. This lets pg_dump use the symbolic names for relevant constants. Discussion: https://postgr.es/m/4834.1507849699@sss.pgh.pa.us
2017-10-14 21:21:39 +02:00
\n\s+\QFINALFUNC = int8_avg,\E
Improve spelling of new FINALFUNC_MODIFY aggregate attribute. I'd used SHARABLE as a value originally, but Peter Eisentraut points out that dictionaries agree that SHAREABLE is the preferred spelling. Run around and change that before it's too late. Discussion: https://postgr.es/m/d2e1afd4-659c-50d6-1b20-7cfd3675e909@2ndquadrant.com
2018-05-21 17:41:42 +02:00
\n\s+\QFINALFUNC_MODIFY = SHAREABLE\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
\n\);/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
exclude_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE CONVERSION dump_test.test_conversion' => {
create_order => 78,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'CREATE DEFAULT CONVERSION dump_test.test_conversion FOR \'LATIN1\' TO \'UTF8\' FROM iso8859_1_to_utf8;',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
qr/^\QCREATE DEFAULT CONVERSION dump_test.test_conversion FOR 'LATIN1' TO 'UTF8' FROM iso8859_1_to_utf8;\E/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE DOMAIN dump_test.us_postal_code' => {
create_order => 29,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE DOMAIN dump_test.us_postal_code AS TEXT
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
COLLATE "C"
DEFAULT \'10014\'
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
CHECK(VALUE ~ \'^\d{5}$\' OR
VALUE ~ \'^\d{5}-\d{4}$\');',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE DOMAIN dump_test.us_postal_code AS text COLLATE pg_catalog."C" DEFAULT '10014'::text\E\n\s+
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\QCONSTRAINT us_postal_code_check CHECK \E
\Q(((VALUE ~ '^\d{5}\E
\$\Q'::text) OR (VALUE ~ '^\d{5}-\d{4}\E\$
\Q'::text)));\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE FUNCTION dump_test.pltestlang_call_handler' => {
create_order => 17,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE FUNCTION dump_test.pltestlang_call_handler()
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
RETURNS LANGUAGE_HANDLER AS \'$libdir/plpgsql\',
\'plpgsql_call_handler\' LANGUAGE C;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE FUNCTION dump_test.pltestlang_call_handler() \E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\QRETURNS language_handler\E
\n\s+\QLANGUAGE c\E
\n\s+AS\ \'\$
\Qlibdir\/plpgsql', 'plpgsql_call_handler';\E
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE FUNCTION dump_test.trigger_func' => {
create_order => 30,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE FUNCTION dump_test.trigger_func()
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
RETURNS trigger LANGUAGE plpgsql
AS $$ BEGIN RETURN NULL; END;$$;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE FUNCTION dump_test.trigger_func() RETURNS trigger\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QLANGUAGE plpgsql\E
\n\s+AS\ \$\$
\Q BEGIN RETURN NULL; END;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
\$\$;/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE FUNCTION dump_test.event_trigger_func' => {
create_order => 32,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE FUNCTION dump_test.event_trigger_func()
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
RETURNS event_trigger LANGUAGE plpgsql
AS $$ BEGIN RETURN; END;$$;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE FUNCTION dump_test.event_trigger_func() RETURNS event_trigger\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QLANGUAGE plpgsql\E
\n\s+AS\ \$\$
\Q BEGIN RETURN; END;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
\$\$;/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE OPERATOR FAMILY dump_test.op_family' => {
create_order => 73,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
'CREATE OPERATOR FAMILY dump_test.op_family USING btree;',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE OPERATOR FAMILY dump_test.op_family USING btree;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE OPERATOR CLASS dump_test.op_class' => {
create_order => 74,
create_sql => 'CREATE OPERATOR CLASS dump_test.op_class
FOR TYPE bigint USING btree FAMILY dump_test.op_family
AS STORAGE bigint,
OPERATOR 1 <(bigint,bigint),
OPERATOR 2 <=(bigint,bigint),
OPERATOR 3 =(bigint,bigint),
OPERATOR 4 >=(bigint,bigint),
OPERATOR 5 >(bigint,bigint),
FUNCTION 1 btint8cmp(bigint,bigint),
FUNCTION 2 btint8sortsupport(internal);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE OPERATOR CLASS dump_test.op_class\E\n\s+
\QFOR TYPE bigint USING btree FAMILY dump_test.op_family AS\E\n\s+
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\QOPERATOR 1 <(bigint,bigint) ,\E\n\s+
\QOPERATOR 2 <=(bigint,bigint) ,\E\n\s+
\QOPERATOR 3 =(bigint,bigint) ,\E\n\s+
\QOPERATOR 4 >=(bigint,bigint) ,\E\n\s+
\QOPERATOR 5 >(bigint,bigint) ,\E\n\s+
\QFUNCTION 1 (bigint, bigint) btint8cmp(bigint,bigint) ,\E\n\s+
\QFUNCTION 2 (bigint, bigint) btint8sortsupport(internal);\E
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add pg_dump test for empty OP class This adds a pg_dump test for an empty operator class. Author: Michael Paquier Discussion: https://postgr.es/m/20181208011142.GU3415@tamriel.snowman.net
2018-12-10 16:13:52 +01:00
'CREATE OPERATOR CLASS dump_test.op_class_empty' => {
create_order => 89,
create_sql => 'CREATE OPERATOR CLASS dump_test.op_class_empty
FOR TYPE bigint USING btree FAMILY dump_test.op_family
AS STORAGE bigint;',
regexp => qr/^
\QCREATE OPERATOR CLASS dump_test.op_class_empty\E\n\s+
\QFOR TYPE bigint USING btree FAMILY dump_test.op_family AS\E\n\s+
\QSTORAGE bigint;\E
/xm,
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => { exclude_dump_test_schema => 1, },
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE EVENT TRIGGER test_event_trigger' => {
create_order => 33,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE EVENT TRIGGER test_event_trigger
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
ON ddl_command_start
Use EXECUTE FUNCTION syntax for triggers more Change pg_dump and ruleutils.c to use the FUNCTION keyword instead of PROCEDURE in trigger and event trigger definitions. This completes the pieces of the transition started in 0a63f996e018ac508c858e87fa39cc254a5db49f that were kept out of PostgreSQL 11 because of the required catversion change. Discussion: https://www.postgresql.org/message-id/381bef53-f7be-29c8-d977-948e389161d6@2ndquadrant.com
2019-02-07 09:01:54 +01:00
EXECUTE FUNCTION dump_test.event_trigger_func();',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
\QCREATE EVENT TRIGGER test_event_trigger \E
\QON ddl_command_start\E
Use EXECUTE FUNCTION syntax for triggers more Change pg_dump and ruleutils.c to use the FUNCTION keyword instead of PROCEDURE in trigger and event trigger definitions. This completes the pieces of the transition started in 0a63f996e018ac508c858e87fa39cc254a5db49f that were kept out of PostgreSQL 11 because of the required catversion change. Discussion: https://www.postgresql.org/message-id/381bef53-f7be-29c8-d977-948e389161d6@2ndquadrant.com
2019-02-07 09:01:54 +01:00
\n\s+\QEXECUTE FUNCTION dump_test.event_trigger_func();\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_post_data => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE TRIGGER test_trigger' => {
create_order => 31,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE TRIGGER test_trigger
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
BEFORE INSERT ON dump_test.test_table
FOR EACH ROW WHEN (NEW.col1 > 10)
Use EXECUTE FUNCTION syntax for triggers more Change pg_dump and ruleutils.c to use the FUNCTION keyword instead of PROCEDURE in trigger and event trigger definitions. This completes the pieces of the transition started in 0a63f996e018ac508c858e87fa39cc254a5db49f that were kept out of PostgreSQL 11 because of the required catversion change. Discussion: https://www.postgresql.org/message-id/381bef53-f7be-29c8-d977-948e389161d6@2ndquadrant.com
2019-02-07 09:01:54 +01:00
EXECUTE FUNCTION dump_test.trigger_func();',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TRIGGER test_trigger BEFORE INSERT ON dump_test.test_table \E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\QFOR EACH ROW WHEN ((new.col1 > 10)) \E
Use EXECUTE FUNCTION syntax for triggers more Change pg_dump and ruleutils.c to use the FUNCTION keyword instead of PROCEDURE in trigger and event trigger definitions. This completes the pieces of the transition started in 0a63f996e018ac508c858e87fa39cc254a5db49f that were kept out of PostgreSQL 11 because of the required catversion change. Discussion: https://www.postgresql.org/message-id/381bef53-f7be-29c8-d977-948e389161d6@2ndquadrant.com
2019-02-07 09:01:54 +01:00
\QEXECUTE FUNCTION dump_test.trigger_func();\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
exclude_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_dump_test_schema => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE TYPE dump_test.planets AS ENUM' => {
create_order => 37,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE TYPE dump_test.planets
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
AS ENUM ( \'venus\', \'earth\', \'mars\' );',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TYPE dump_test.planets AS ENUM (\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+'venus',
\n\s+'earth',
\n\s+'mars'
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
\n\);/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_dump_test_schema => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE TYPE dump_test.planets AS ENUM pg_upgrade' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TYPE dump_test.planets AS ENUM (\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\);.*^
\QALTER TYPE dump_test.planets ADD VALUE 'venus';\E
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'earth';\E
\n.*^
\QALTER TYPE dump_test.planets ADD VALUE 'mars';\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
\n/xms,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { binary_upgrade => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TYPE dump_test.textrange AS RANGE' => {
create_order => 38,
create_sql => 'CREATE TYPE dump_test.textrange
AS RANGE (subtype=text, collation="C");',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TYPE dump_test.textrange AS RANGE (\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n\s+\Qsubtype = text,\E
\n\s+\Qcollation = pg_catalog."C"\E
\n\);/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TYPE dump_test.int42' => {
create_order => 39,
create_sql => 'CREATE TYPE dump_test.int42;',
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
regexp => qr/^\QCREATE TYPE dump_test.int42;\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1' => {
create_order => 80,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'CREATE TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1 (copy=english);',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1 (\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QPARSER = pg_catalog."default" );\E/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1 ...' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR asciiword WITH english_stem;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR word WITH english_stem;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR numword WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR email WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR url WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR host WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR sfloat WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR version WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR hword_numpart WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR hword_part WITH english_stem;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR hword_asciipart WITH english_stem;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR numhword WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR asciihword WITH english_stem;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR hword WITH english_stem;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR url_path WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR file WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR "float" WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR "int" WITH simple;\E\n
\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TEXT SEARCH CONFIGURATION dump_test.alt_ts_conf1\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QADD MAPPING FOR uint WITH simple;\E\n
\n
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TEXT SEARCH TEMPLATE dump_test.alt_ts_temp1' => {
create_order => 81,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'CREATE TEXT SEARCH TEMPLATE dump_test.alt_ts_temp1 (lexize=dsimple_lexize);',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TEXT SEARCH TEMPLATE dump_test.alt_ts_temp1 (\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QLEXIZE = dsimple_lexize );\E/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TEXT SEARCH PARSER dump_test.alt_ts_prs1' => {
create_order => 82,
create_sql => 'CREATE TEXT SEARCH PARSER dump_test.alt_ts_prs1
(start = prsd_start, gettoken = prsd_nexttoken, end = prsd_end, lextypes = prsd_lextype);',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TEXT SEARCH PARSER dump_test.alt_ts_prs1 (\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QSTART = prsd_start,\E\n
\s+\QGETTOKEN = prsd_nexttoken,\E\n
\s+\QEND = prsd_end,\E\n
\s+\QLEXTYPES = prsd_lextype );\E\n
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TEXT SEARCH DICTIONARY dump_test.alt_ts_dict1' => {
create_order => 83,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'CREATE TEXT SEARCH DICTIONARY dump_test.alt_ts_dict1 (template=simple);',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TEXT SEARCH DICTIONARY dump_test.alt_ts_dict1 (\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\QTEMPLATE = pg_catalog.simple );\E\n
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE FUNCTION dump_test.int42_in' => {
create_order => 40,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE FUNCTION dump_test.int42_in(cstring)
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
RETURNS dump_test.int42 AS \'int4in\'
LANGUAGE internal STRICT IMMUTABLE;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE FUNCTION dump_test.int42_in(cstring) RETURNS dump_test.int42\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QLANGUAGE internal IMMUTABLE STRICT\E
\n\s+AS\ \$\$int4in\$\$;
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE FUNCTION dump_test.int42_out' => {
create_order => 41,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE FUNCTION dump_test.int42_out(dump_test.int42)
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
RETURNS cstring AS \'int4out\'
LANGUAGE internal STRICT IMMUTABLE;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE FUNCTION dump_test.int42_out(dump_test.int42) RETURNS cstring\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QLANGUAGE internal IMMUTABLE STRICT\E
\n\s+AS\ \$\$int4out\$\$;
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
SQL procedures This adds a new object type "procedure" that is similar to a function but does not have a return type and is invoked by the new CALL statement instead of SELECT or similar. This implementation is aligned with the SQL standard and compatible with or similar to other SQL implementations. This commit adds new commands CALL, CREATE/ALTER/DROP PROCEDURE, as well as ALTER/DROP ROUTINE that can refer to either a function or a procedure (or an aggregate function, as an extension to SQL). There is also support for procedures in various utility commands such as COMMENT and GRANT, as well as support in pg_dump and psql. Support for defining procedures is available in all the languages supplied by the core distribution. While this commit is mainly syntax sugar around existing functionality, future features will rely on having procedures as a separate object type. Reviewed-by: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2017-11-30 14:46:13 +01:00
Create the infrastructure for planner support functions. Rename/repurpose pg_proc.protransform as "prosupport". The idea is still that it names an internal function that provides knowledge to the planner about the behavior of the function it's attached to; but redesign the API specification so that it's not limited to doing just one thing, but can support an extensible set of requests. The original purpose of simplifying a function call is handled by the first request type to be invented, SupportRequestSimplify. Adjust all the existing transform functions to handle this API, and rename them fron "xxx_transform" to "xxx_support" to reflect the potential generalization of what they do. (Since we never previously provided any way for extensions to add transform functions, this change doesn't create an API break for them.) Also add DDL and pg_dump support for attaching a support function to a user-defined function. Unfortunately, DDL access has to be restricted to superusers, at least for now; but seeing that support functions will pretty much have to be written in C, that limitation is just theoretical. (This support is untested in this patch, but a follow-on patch will add cases that exercise it.) Discussion: https://postgr.es/m/15193.1548028093@sss.pgh.pa.us
2019-02-10 00:08:48 +01:00
'CREATE FUNCTION ... SUPPORT' => {
create_order => 41,
create_sql =>
'CREATE FUNCTION dump_test.func_with_support() RETURNS int LANGUAGE sql AS $$ SELECT 1 $$ SUPPORT varchar_support;',
regexp => qr/^
\QCREATE FUNCTION dump_test.func_with_support() RETURNS integer\E
\n\s+\QLANGUAGE sql SUPPORT varchar_support\E
\n\s+AS\ \$\$\Q SELECT 1 \E\$\$;
/xm,
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => { exclude_dump_test_schema => 1, },
},
SQL procedures This adds a new object type "procedure" that is similar to a function but does not have a return type and is invoked by the new CALL statement instead of SELECT or similar. This implementation is aligned with the SQL standard and compatible with or similar to other SQL implementations. This commit adds new commands CALL, CREATE/ALTER/DROP PROCEDURE, as well as ALTER/DROP ROUTINE that can refer to either a function or a procedure (or an aggregate function, as an extension to SQL). There is also support for procedures in various utility commands such as COMMENT and GRANT, as well as support in pg_dump and psql. Support for defining procedures is available in all the languages supplied by the core distribution. While this commit is mainly syntax sugar around existing functionality, future features will rely on having procedures as a separate object type. Reviewed-by: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2017-11-30 14:46:13 +01:00
'CREATE PROCEDURE dump_test.ptest1' => {
create_order => 41,
create_sql => 'CREATE PROCEDURE dump_test.ptest1(a int)
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
LANGUAGE SQL AS $$ INSERT INTO dump_test.test_table (col1) VALUES (a) $$;',
regexp => qr/^
\QCREATE PROCEDURE dump_test.ptest1(a integer)\E
\n\s+\QLANGUAGE sql\E
\n\s+AS\ \$\$\Q INSERT INTO dump_test.test_table (col1) VALUES (a) \E\$\$;
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE TYPE dump_test.int42 populated' => {
create_order => 42,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE TYPE dump_test.int42 (
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
internallength = 4,
input = dump_test.int42_in,
output = dump_test.int42_out,
alignment = int4,
default = 42,
passedbyvalue);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TYPE dump_test.int42 (\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QINTERNALLENGTH = 4,\E
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\n\s+\QINPUT = dump_test.int42_in,\E
\n\s+\QOUTPUT = dump_test.int42_out,\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QDEFAULT = '42',\E
\n\s+\QALIGNMENT = int4,\E
\n\s+\QSTORAGE = plain,\E
\n\s+PASSEDBYVALUE\n\);
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE TYPE dump_test.composite' => {
create_order => 43,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE TYPE dump_test.composite AS (
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
f1 int,
f2 dump_test.int42
);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TYPE dump_test.composite AS (\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\Qf1 integer,\E
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\n\s+\Qf2 dump_test.int42\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\);
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TYPE dump_test.undefined' => {
create_order => 39,
create_sql => 'CREATE TYPE dump_test.undefined;',
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
regexp => qr/^\QCREATE TYPE dump_test.undefined;\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE FOREIGN DATA WRAPPER dummy' => {
create_order => 35,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE FOREIGN DATA WRAPPER dummy;',
regexp => qr/CREATE FOREIGN DATA WRAPPER dummy;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_pre_data => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE SERVER s1 FOREIGN DATA WRAPPER dummy' => {
create_order => 36,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE SERVER s1 FOREIGN DATA WRAPPER dummy;',
regexp => qr/CREATE SERVER s1 FOREIGN DATA WRAPPER dummy;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_pre_data => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE FOREIGN TABLE dump_test.foreign_table SERVER s1' => {
create_order => 88,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'CREATE FOREIGN TABLE dump_test.foreign_table (c1 int options (column_name \'col1\'))
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
SERVER s1 OPTIONS (schema_name \'x1\');',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp => qr/
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE FOREIGN TABLE dump_test.foreign_table (\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\Qc1 integer\E\n
\Q)\E\n
\QSERVER s1\E\n
\QOPTIONS (\E\n
\s+\Qschema_name 'x1'\E\n
\Q);\E\n
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE USER MAPPING FOR regress_dump_test_role SERVER s1' => {
create_order => 86,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
'CREATE USER MAPPING FOR regress_dump_test_role SERVER s1;',
regexp =>
qr/CREATE USER MAPPING FOR regress_dump_test_role SERVER s1;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_pre_data => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Fix dumping of casts and transforms using built-in functions In pg_dump.c dumpCast() and dumpTransform(), we would happily ignore the cast or transform if it happened to use a built-in function because we weren't including the information about built-in functions when querying pg_proc from getFuncs(). Modify the query in getFuncs() to also gather information about functions which are used by user-defined casts and transforms (where "user-defined" means "has an OID >= FirstNormalObjectId"). This also adds to the TAP regression tests for 9.6 and master to cover these types of objects. Back-patch all the way for casts, back to 9.5 for transforms. Discussion: https://www.postgresql.org/message-id/flat/20160504183952.GE10850%40tamriel.snowman.net
2016-12-21 19:47:06 +01:00
'CREATE TRANSFORM FOR int' => {
create_order => 34,
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
create_sql =>
Solve cross-version-upgrade testing problem induced by 1fb57af92. Renaming varchar_transform to varchar_support had a side effect I hadn't foreseen: the core regression tests leave around a transform object that relies on that function, so the name change breaks cross-version upgrade tests, because the name used in the older branches doesn't match. Since the dependency on varchar_transform was chosen with the aid of a dartboard anyway (it would surely not work as a language transform support function), fix by just choosing a different random builtin function with the right signature. Also add some comments explaining why this isn't horribly unsafe. I chose to make the same substitution in a couple of other copied-and-pasted test cases, for consistency, though those aren't directly contributing to the testing problem. Per buildfarm. Back-patch, else it doesn't fix the problem.
2019-02-10 03:02:06 +01:00
'CREATE TRANSFORM FOR int LANGUAGE SQL (FROM SQL WITH FUNCTION prsd_lextype(internal), TO SQL WITH FUNCTION int4recv(internal));',
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
regexp =>
Solve cross-version-upgrade testing problem induced by 1fb57af92. Renaming varchar_transform to varchar_support had a side effect I hadn't foreseen: the core regression tests leave around a transform object that relies on that function, so the name change breaks cross-version upgrade tests, because the name used in the older branches doesn't match. Since the dependency on varchar_transform was chosen with the aid of a dartboard anyway (it would surely not work as a language transform support function), fix by just choosing a different random builtin function with the right signature. Also add some comments explaining why this isn't horribly unsafe. I chose to make the same substitution in a couple of other copied-and-pasted test cases, for consistency, though those aren't directly contributing to the testing problem. Per buildfarm. Back-patch, else it doesn't fix the problem.
2019-02-10 03:02:06 +01:00
qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog\.prsd_lextype\(internal\), TO SQL WITH FUNCTION pg_catalog\.int4recv\(internal\)\);/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_pre_data => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE LANGUAGE pltestlang' => {
create_order => 18,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE LANGUAGE pltestlang
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
HANDLER dump_test.pltestlang_call_handler;',
regexp => qr/^
\QCREATE PROCEDURAL LANGUAGE pltestlang \E
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QHANDLER dump_test.pltestlang_call_handler;\E
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
/xm,
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE MATERIALIZED VIEW matview' => {
create_order => 20,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE MATERIALIZED VIEW dump_test.matview (col1) AS
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
SELECT col1 FROM dump_test.test_table;',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE MATERIALIZED VIEW dump_test.matview AS\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QSELECT test_table.col1\E
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\n\s+\QFROM dump_test.test_table\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QWITH NO DATA;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE MATERIALIZED VIEW matview_second' => {
create_order => 21,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE MATERIALIZED VIEW
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
dump_test.matview_second (col1) AS
SELECT * FROM dump_test.matview;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE MATERIALIZED VIEW dump_test.matview_second AS\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QSELECT matview.col1\E
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\n\s+\QFROM dump_test.matview\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QWITH NO DATA;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE MATERIALIZED VIEW matview_third' => {
create_order => 58,
create_sql => 'CREATE MATERIALIZED VIEW
dump_test.matview_third (col1) AS
SELECT * FROM dump_test.matview_second WITH NO DATA;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE MATERIALIZED VIEW dump_test.matview_third AS\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n\s+\QSELECT matview_second.col1\E
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\n\s+\QFROM dump_test.matview_second\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n\s+\QWITH NO DATA;\E
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE MATERIALIZED VIEW matview_fourth' => {
create_order => 59,
create_sql => 'CREATE MATERIALIZED VIEW
dump_test.matview_fourth (col1) AS
SELECT * FROM dump_test.matview_third WITH NO DATA;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE MATERIALIZED VIEW dump_test.matview_fourth AS\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n\s+\QSELECT matview_third.col1\E
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\n\s+\QFROM dump_test.matview_third\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n\s+\QWITH NO DATA;\E
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE POLICY p1 ON test_table' => {
create_order => 22,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE POLICY p1 ON dump_test.test_table
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
USING (true)
WITH CHECK (true);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE POLICY p1 ON dump_test.test_table \E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\QUSING (true) WITH CHECK (true);\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE POLICY p2 ON test_table FOR SELECT' => {
create_order => 24,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE POLICY p2 ON dump_test.test_table
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
FOR SELECT TO regress_dump_test_role USING (true);',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE POLICY p2 ON dump_test.test_table FOR SELECT TO regress_dump_test_role \E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\QUSING (true);\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE POLICY p3 ON test_table FOR INSERT' => {
create_order => 25,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE POLICY p3 ON dump_test.test_table
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
FOR INSERT TO regress_dump_test_role WITH CHECK (true);',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE POLICY p3 ON dump_test.test_table FOR INSERT \E
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
\QTO regress_dump_test_role WITH CHECK (true);\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE POLICY p4 ON test_table FOR UPDATE' => {
create_order => 26,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE POLICY p4 ON dump_test.test_table FOR UPDATE
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
TO regress_dump_test_role USING (true) WITH CHECK (true);',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE POLICY p4 ON dump_test.test_table FOR UPDATE TO regress_dump_test_role \E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\QUSING (true) WITH CHECK (true);\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE POLICY p5 ON test_table FOR DELETE' => {
create_order => 27,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE POLICY p5 ON dump_test.test_table
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
FOR DELETE TO regress_dump_test_role USING (true);',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE POLICY p5 ON dump_test.test_table FOR DELETE \E
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
\QTO regress_dump_test_role USING (true);\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
Add support for restrictive RLS policies We have had support for restrictive RLS policies since 9.5, but they were only available through extensions which use the appropriate hooks. This adds support into the grammer, catalog, psql and pg_dump for restrictive RLS policies, thus reducing the cases where an extension is necessary. In passing, also move away from using "AND"d and "OR"d in comments. As pointed out by Alvaro, it's not really appropriate to attempt to make verbs out of "AND" and "OR", so reword those comments which attempted to. Reviewed By: Jeevan Chalke, Dean Rasheed Discussion: https://postgr.es/m/20160901063404.GY4028@tamriel.snowman.net
2016-12-05 21:50:55 +01:00
'CREATE POLICY p6 ON test_table AS RESTRICTIVE' => {
create_order => 27,
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
create_sql => 'CREATE POLICY p6 ON dump_test.test_table AS RESTRICTIVE
Add support for restrictive RLS policies We have had support for restrictive RLS policies since 9.5, but they were only available through extensions which use the appropriate hooks. This adds support into the grammer, catalog, psql and pg_dump for restrictive RLS policies, thus reducing the cases where an extension is necessary. In passing, also move away from using "AND"d and "OR"d in comments. As pointed out by Alvaro, it's not really appropriate to attempt to make verbs out of "AND" and "OR", so reword those comments which attempted to. Reviewed By: Jeevan Chalke, Dean Rasheed Discussion: https://postgr.es/m/20160901063404.GY4028@tamriel.snowman.net
2016-12-05 21:50:55 +01:00
USING (false);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE POLICY p6 ON dump_test.test_table AS RESTRICTIVE \E
Add support for restrictive RLS policies We have had support for restrictive RLS policies since 9.5, but they were only available through extensions which use the appropriate hooks. This adds support into the grammer, catalog, psql and pg_dump for restrictive RLS policies, thus reducing the cases where an extension is necessary. In passing, also move away from using "AND"d and "OR"d in comments. As pointed out by Alvaro, it's not really appropriate to attempt to make verbs out of "AND" and "OR", so reword those comments which attempted to. Reviewed By: Jeevan Chalke, Dean Rasheed Discussion: https://postgr.es/m/20160901063404.GY4028@tamriel.snowman.net
2016-12-05 21:50:55 +01:00
\QUSING (false);\E
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
Add support for restrictive RLS policies We have had support for restrictive RLS policies since 9.5, but they were only available through extensions which use the appropriate hooks. This adds support into the grammer, catalog, psql and pg_dump for restrictive RLS policies, thus reducing the cases where an extension is necessary. In passing, also move away from using "AND"d and "OR"d in comments. As pointed out by Alvaro, it's not really appropriate to attempt to make verbs out of "AND" and "OR", so reword those comments which attempted to. Reviewed By: Jeevan Chalke, Dean Rasheed Discussion: https://postgr.es/m/20160901063404.GY4028@tamriel.snowman.net
2016-12-05 21:50:55 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE PUBLICATION pub1' => {
create_order => 50,
create_sql => 'CREATE PUBLICATION pub1;',
regexp => qr/^
Logical replication support for TRUNCATE Update the built-in logical replication system to make use of the previously added logical decoding for TRUNCATE support. Add the required truncate callback to pgoutput and a new logical replication protocol message. Publications get a new attribute to determine whether to replicate truncate actions. When updating a publication via pg_dump from an older version, this is not set, thus preserving the previous behavior. Author: Simon Riggs <simon@2ndquadrant.com> Author: Marco Nenciarini <marco.nenciarini@2ndquadrant.it> Author: Peter Eisentraut <peter.eisentraut@2ndquadrant.com> Reviewed-by: Petr Jelinek <petr.jelinek@2ndquadrant.com> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
2018-04-07 17:24:53 +02:00
\QCREATE PUBLICATION pub1 WITH (publish = 'insert, update, delete, truncate');\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_post_data => 1, },
},
Logical replication - Add PUBLICATION catalogs and DDL - Add SUBSCRIPTION catalog and DDL - Define logical replication protocol and output plugin - Add logical replication workers From: Petr Jelinek <petr@2ndquadrant.com> Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Erik Rijkers <er@xs4all.nl> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-01-19 18:00:00 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE PUBLICATION pub2' => {
Logical replication - Add PUBLICATION catalogs and DDL - Add SUBSCRIPTION catalog and DDL - Define logical replication protocol and output plugin - Add logical replication workers From: Petr Jelinek <petr@2ndquadrant.com> Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Erik Rijkers <er@xs4all.nl> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-01-19 18:00:00 +01:00
create_order => 50,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
create_sql => 'CREATE PUBLICATION pub2
FOR ALL TABLES
Rework the options syntax for logical replication commands For CREATE/ALTER PUBLICATION/SUBSCRIPTION, use similar option style as other statements that use a WITH clause for options. Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-05-12 14:57:01 +02:00
WITH (publish = \'\');',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp => qr/^
Rework the options syntax for logical replication commands For CREATE/ALTER PUBLICATION/SUBSCRIPTION, use similar option style as other statements that use a WITH clause for options. Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-05-12 14:57:01 +02:00
\QCREATE PUBLICATION pub2 FOR ALL TABLES WITH (publish = '');\E
Logical replication - Add PUBLICATION catalogs and DDL - Add SUBSCRIPTION catalog and DDL - Define logical replication protocol and output plugin - Add logical replication workers From: Petr Jelinek <petr@2ndquadrant.com> Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Erik Rijkers <er@xs4all.nl> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-01-19 18:00:00 +01:00
/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_post_data => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE SUBSCRIPTION sub1' => {
create_order => 50,
create_sql => 'CREATE SUBSCRIPTION sub1
CONNECTION \'dbname=doesnotexist\' PUBLICATION pub1
Rework the options syntax for logical replication commands For CREATE/ALTER PUBLICATION/SUBSCRIPTION, use similar option style as other statements that use a WITH clause for options. Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-05-12 14:57:01 +02:00
WITH (connect = false);',
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp => qr/^
Rework the options syntax for logical replication commands For CREATE/ALTER PUBLICATION/SUBSCRIPTION, use similar option style as other statements that use a WITH clause for options. Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-05-12 14:57:01 +02:00
\QCREATE SUBSCRIPTION sub1 CONNECTION 'dbname=doesnotexist' PUBLICATION pub1 WITH (connect = false, slot_name = 'sub1');\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { %full_runs, section_post_data => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
Logical replication - Add PUBLICATION catalogs and DDL - Add SUBSCRIPTION catalog and DDL - Define logical replication protocol and output plugin - Add logical replication workers From: Petr Jelinek <petr@2ndquadrant.com> Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Erik Rijkers <er@xs4all.nl> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-01-19 18:00:00 +01:00
'ALTER PUBLICATION pub1 ADD TABLE test_table' => {
create_order => 51,
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
create_sql =>
'ALTER PUBLICATION pub1 ADD TABLE dump_test.test_table;',
Logical replication - Add PUBLICATION catalogs and DDL - Add SUBSCRIPTION catalog and DDL - Define logical replication protocol and output plugin - Add logical replication workers From: Petr Jelinek <petr@2ndquadrant.com> Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Erik Rijkers <er@xs4all.nl> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-01-19 18:00:00 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER PUBLICATION pub1 ADD TABLE ONLY dump_test.test_table;\E
Logical replication - Add PUBLICATION catalogs and DDL - Add SUBSCRIPTION catalog and DDL - Define logical replication protocol and output plugin - Add logical replication workers From: Petr Jelinek <petr@2ndquadrant.com> Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Erik Rijkers <er@xs4all.nl> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-01-19 18:00:00 +01:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_post_data => 1, },
Logical replication - Add PUBLICATION catalogs and DDL - Add SUBSCRIPTION catalog and DDL - Define logical replication protocol and output plugin - Add logical replication workers From: Petr Jelinek <petr@2ndquadrant.com> Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Erik Rijkers <er@xs4all.nl> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-01-19 18:00:00 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
pg_dump: Fix dumping of publications Dumping a publication with more than one table crashed pg_dump. patch by Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, test by me
2017-03-10 21:31:47 +01:00
'ALTER PUBLICATION pub1 ADD TABLE test_second_table' => {
create_order => 52,
create_sql =>
'ALTER PUBLICATION pub1 ADD TABLE dump_test.test_second_table;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER PUBLICATION pub1 ADD TABLE ONLY dump_test.test_second_table;\E
pg_dump: Fix dumping of publications Dumping a publication with more than one table crashed pg_dump. patch by Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, test by me
2017-03-10 21:31:47 +01:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_post_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Logical replication - Add PUBLICATION catalogs and DDL - Add SUBSCRIPTION catalog and DDL - Define logical replication protocol and output plugin - Add logical replication workers From: Petr Jelinek <petr@2ndquadrant.com> Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Erik Rijkers <er@xs4all.nl> Reviewed-by: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
2017-01-19 18:00:00 +01:00
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
'CREATE SCHEMA public' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^CREATE SCHEMA public;/m,
Improve pg_dump's handling of "special" built-in objects. We had some pretty ad-hoc handling of the public schema and the plpgsql extension, which are both presumed to exist in template0 but might be modified or deleted in the database being dumped. Up to now, by default pg_dump would emit a CREATE EXTENSION IF NOT EXISTS command as well as a COMMENT command for plpgsql. The usefulness of the former is questionable, and the latter caused annoying errors in non-superuser dump/restore scenarios. Let's instead install a rule that built-in extensions (identified by having low-numbered OIDs) are not to be dumped. We were doing it that way already in binary-upgrade mode, so this just makes regular mode behave the same. It remains true that if someone has installed a non-default ACL on the plpgsql language, that will get dumped thanks to the pg_init_privs mechanism. This is more consistent with the handling of built-in objects of other kinds. Also, change the very ad-hoc mechanism that was used to avoid dumping creation and comment commands for the public schema. Instead of hardwiring a test in _printTocEntry(), make use of the DUMP_COMPONENT_ infrastructure to mark that schema up-front about what we want to do with it. This has the visible effect that the public schema won't be mentioned in the output at all, except for updating its ACL if it has a non-default ACL. Previously, while it was normally not mentioned, --clean mode would drop and recreate it, again causing headaches for non-superuser usage. This change likewise makes the public schema less special and more like other built-in objects. If plpgsql, or the public schema, has been removed entirely in the source DB, that situation won't be reproduced in the destination ... but that was true before. Discussion: https://postgr.es/m/29048.1516812451@sss.pgh.pa.us
2018-01-25 19:54:42 +01:00
# this shouldn't ever get emitted anymore
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE SCHEMA dump_test' => {
create_order => 2,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE SCHEMA dump_test;',
regexp => qr/^CREATE SCHEMA dump_test;/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE SCHEMA dump_test_second_schema' => {
create_order => 9,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE SCHEMA dump_test_second_schema;',
regexp => qr/^CREATE SCHEMA dump_test_second_schema;/m,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
role => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE TABLE test_table' => {
create_order => 3,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE TABLE dump_test.test_table (
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
col1 serial primary key,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
col2 text,
col3 text,
col4 text,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
CHECK (col1 <= 1000)
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
) WITH (autovacuum_enabled = false, fillfactor=80);',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TABLE dump_test.test_table (\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\Qcol1 integer NOT NULL,\E\n
\s+\Qcol2 text,\E\n
\s+\Qcol3 text,\E\n
\s+\Qcol4 text,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_test_table => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE TABLE fk_reference_test_table' => {
create_order => 21,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE TABLE dump_test.fk_reference_test_table (
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
col1 int primary key references dump_test.test_table
);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TABLE dump_test.fk_reference_test_table (\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\Qcol1 integer NOT NULL\E
\n\);
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE TABLE test_second_table' => {
create_order => 6,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'CREATE TABLE dump_test.test_second_table (
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
col1 int,
col2 text
);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TABLE dump_test.test_second_table (\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 text\E
\n\);
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TABLE measurement PARTITIONED BY' => {
create_order => 90,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql => 'CREATE TABLE dump_test.measurement (
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
city_id int not null,
logdate date not null,
peaktemp int,
unitsales int
) PARTITION BY RANGE (logdate);',
regexp => qr/^
\Q-- Name: measurement;\E.*\n
\Q--\E\n\n
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TABLE dump_test.measurement (\E\n
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\s+\Qcity_id integer NOT NULL,\E\n
\s+\Qlogdate date NOT NULL,\E\n
\s+\Qpeaktemp integer,\E\n
\s+\Qunitsales integer\E\n
\)\n
\QPARTITION BY RANGE (logdate);\E\n
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
exclude_dump_test_schema => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TABLE measurement_y2006m2 PARTITION OF' => {
create_order => 91,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
'CREATE TABLE dump_test_second_schema.measurement_y2006m2
Add additional partition tests to pg_dump This adds a few tests for non-inherited constraints. Author: Amit Langote Discussion: https://postgr.es/m/20181208001735.GT3415%40tamriel.snowman.net
2018-12-10 15:46:36 +01:00
PARTITION OF dump_test.measurement (
unitsales DEFAULT 0 CHECK (unitsales >= 0)
)
FOR VALUES FROM (\'2006-02-01\') TO (\'2006-03-01\');',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
\Q-- Name: measurement_y2006m2;\E.*\n
\Q--\E\n\n
Add additional partition tests to pg_dump This adds a few tests for non-inherited constraints. Author: Amit Langote Discussion: https://postgr.es/m/20181208001735.GT3415%40tamriel.snowman.net
2018-12-10 15:46:36 +01:00
\QCREATE TABLE dump_test_second_schema.measurement_y2006m2 PARTITION OF dump_test.measurement (\E\n
\s+\QCONSTRAINT measurement_y2006m2_unitsales_check CHECK ((unitsales >= 0))\E\n
\)\n
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
\QFOR VALUES FROM ('2006-02-01') TO ('2006-03-01');\E\n
/xm,
like => {
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
role => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
unlike => { binary_upgrade => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TABLE test_fourth_table_zero_col' => {
create_order => 6,
create_sql => 'CREATE TABLE dump_test.test_fourth_table (
);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TABLE dump_test.test_fourth_table (\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n\);
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE TABLE test_fifth_table' => {
create_order => 53,
create_sql => 'CREATE TABLE dump_test.test_fifth_table (
col1 integer,
col2 boolean,
col3 boolean,
col4 bit(5),
col5 float8
);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TABLE dump_test.test_fifth_table (\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
\n\s+\Qcol4 bit(5),\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
'CREATE TABLE test_table_identity' => {
create_order => 3,
create_sql => 'CREATE TABLE dump_test.test_table_identity (
col1 int generated always as identity primary key,
col2 text
);',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE TABLE dump_test.test_table_identity (\E\n
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
\s+\Qcol1 integer NOT NULL,\E\n
\s+\Qcol2 text\E\n
\);
.*
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
\s+\QSEQUENCE NAME dump_test.test_table_identity_col1_seq\E\n
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
\s+\QSTART WITH 1\E\n
\s+\QINCREMENT BY 1\E\n
\s+\QNO MINVALUE\E\n
\s+\QNO MAXVALUE\E\n
\s+\QCACHE 1\E\n
\);
/xms,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
Generated columns This is an SQL-standard feature that allows creating columns that are computed from expressions rather than assigned, similar to a view or materialized view but on a column basis. This implements one kind of generated column: stored (computed on write). Another kind, virtual (computed on read), is planned for the future, and some room is left for it. Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/b151f851-4019-bdb1-699e-ebab07d2f40a@2ndquadrant.com
2019-03-30 08:13:09 +01:00
'CREATE TABLE test_table_generated' => {
create_order => 3,
create_sql => 'CREATE TABLE dump_test.test_table_generated (
col1 int primary key,
col2 int generated always as (col1 * 2) stored
);',
regexp => qr/^
\QCREATE TABLE dump_test.test_table_generated (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
\s+\Qcol2 integer GENERATED ALWAYS AS ((col1 * 2)) STORED\E\n
\);
/xms,
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
unlike => { exclude_dump_test_schema => 1, },
},
Include ALTER INDEX SET STATISTICS in pg_dump The new grammar pattern of ALTER INDEX SET STATISTICS able to use column numbers on top of the existing column names introduced by commit 5b6d13e forgot to add support for the feature in pg_dump, so defining statistics on index columns was missing from the dumps, potentially causing silent planning problems with a subsequent restore. pg_dump ought to not use column names in what it generates as these are automatically generated by the server and could conflict with real relation attributes with matching patterns. "expr" and "exprN", N incremented automatically after the creation of the first one, are used as default attribute names for index expressions, and that could easily match what is defined in other relations, causing the dumps to fail if some of those attributes are renamed at some point. So to avoid any problems, the new grammar with column numbers gets used. Reported-by: Ronan Dunklau Author: Michael Paquier Reviewed-by: Tom Lane, Adrien Nayrat, Amul Sul Discussion: https://postgr.es/m/CAARsnT3UQ4V=yDNW468w8RqHfYiY9mpn2r_c5UkBJ97NAApUEw@mail.gmail.com Backpatch-through: 11, where the new syntax has been introduced.
2018-12-18 01:28:16 +01:00
'CREATE TABLE table_with_stats' => {
create_order => 98,
create_sql => 'CREATE TABLE dump_test.table_index_stats (
col1 int,
col2 int,
col3 int);
CREATE INDEX index_with_stats
ON dump_test.table_index_stats
((col1 + 1), col1, (col2 + 1), (col3 + 1));
ALTER INDEX dump_test.index_with_stats
ALTER COLUMN 1 SET STATISTICS 400;
ALTER INDEX dump_test.index_with_stats
ALTER COLUMN 3 SET STATISTICS 500;',
regexp => qr/^
\QALTER INDEX dump_test.index_with_stats ALTER COLUMN 1 SET STATISTICS 400;\E\n
\QALTER INDEX dump_test.index_with_stats ALTER COLUMN 3 SET STATISTICS 500;\E\n
/xms,
like =>
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
unlike => { exclude_dump_test_schema => 1, },
},
Add pg_dump tests for CREATE STATISTICS CREATE STATISTICS pg_dump support code was not covered at all by previous tests. Discussion: https://postgr.es/m/20170503172746.rwftidszir67sgk7@alvherre.pgsql
2017-05-03 20:52:00 +02:00
'CREATE STATISTICS extended_stats_no_options' => {
create_order => 97,
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_no_options
Change CREATE STATISTICS syntax Previously, we had the WITH clause in the middle of the command, where you'd specify both generic options as well as statistic types. Few people liked this, so this commit changes it to remove the WITH keyword from that clause and makes it accept statistic types only. (We currently don't have any generic options, but if we invent in the future, we will gain a new WITH clause, probably at the end of the command). Also, the column list is now specified without parens, which makes the whole command look more similar to a SELECT command. This change will let us expand the command to supporting expressions (not just columns names) as well as multiple tables and their join conditions. Tom added lots of code comments and fixed some parts of the CREATE STATISTICS reference page, too; more changes in this area are forthcoming. He also fixed a potential problem in the alter_generic regression test, reducing verbosity on a cascaded drop to avoid dependency on message ordering, as we do in other tests. Tom also closed a security bug: we documented that table ownership was required in order to create a statistics object on it, but didn't actually implement it. Implement tab-completion for statistics objects. This can stand some more improvement. Authors: Alvaro Herrera, with lots of cleanup by Tom Lane Discussion: https://postgr.es/m/20170420212426.ltvgyhnefvhixm6i@alvherre.pgsql
2017-05-12 19:59:23 +02:00
ON col1, col2 FROM dump_test.test_fifth_table',
Add pg_dump tests for CREATE STATISTICS CREATE STATISTICS pg_dump support code was not covered at all by previous tests. Discussion: https://postgr.es/m/20170503172746.rwftidszir67sgk7@alvherre.pgsql
2017-05-03 20:52:00 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE STATISTICS dump_test.test_ext_stats_no_options ON col1, col2 FROM dump_test.test_fifth_table;\E
Add pg_dump tests for CREATE STATISTICS CREATE STATISTICS pg_dump support code was not covered at all by previous tests. Discussion: https://postgr.es/m/20170503172746.rwftidszir67sgk7@alvherre.pgsql
2017-05-03 20:52:00 +02:00
/xms,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Add pg_dump tests for CREATE STATISTICS CREATE STATISTICS pg_dump support code was not covered at all by previous tests. Discussion: https://postgr.es/m/20170503172746.rwftidszir67sgk7@alvherre.pgsql
2017-05-03 20:52:00 +02:00
'CREATE STATISTICS extended_stats_options' => {
create_order => 97,
Change CREATE STATISTICS syntax Previously, we had the WITH clause in the middle of the command, where you'd specify both generic options as well as statistic types. Few people liked this, so this commit changes it to remove the WITH keyword from that clause and makes it accept statistic types only. (We currently don't have any generic options, but if we invent in the future, we will gain a new WITH clause, probably at the end of the command). Also, the column list is now specified without parens, which makes the whole command look more similar to a SELECT command. This change will let us expand the command to supporting expressions (not just columns names) as well as multiple tables and their join conditions. Tom added lots of code comments and fixed some parts of the CREATE STATISTICS reference page, too; more changes in this area are forthcoming. He also fixed a potential problem in the alter_generic regression test, reducing verbosity on a cascaded drop to avoid dependency on message ordering, as we do in other tests. Tom also closed a security bug: we documented that table ownership was required in order to create a statistics object on it, but didn't actually implement it. Implement tab-completion for statistics objects. This can stand some more improvement. Authors: Alvaro Herrera, with lots of cleanup by Tom Lane Discussion: https://postgr.es/m/20170420212426.ltvgyhnefvhixm6i@alvherre.pgsql
2017-05-12 19:59:23 +02:00
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_opts
(ndistinct) ON col1, col2 FROM dump_test.test_fifth_table',
Add pg_dump tests for CREATE STATISTICS CREATE STATISTICS pg_dump support code was not covered at all by previous tests. Discussion: https://postgr.es/m/20170503172746.rwftidszir67sgk7@alvherre.pgsql
2017-05-03 20:52:00 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE STATISTICS dump_test.test_ext_stats_opts (ndistinct) ON col1, col2 FROM dump_test.test_fifth_table;\E
Add pg_dump tests for CREATE STATISTICS CREATE STATISTICS pg_dump support code was not covered at all by previous tests. Discussion: https://postgr.es/m/20170503172746.rwftidszir67sgk7@alvherre.pgsql
2017-05-03 20:52:00 +02:00
/xms,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Add pg_dump tests for CREATE STATISTICS CREATE STATISTICS pg_dump support code was not covered at all by previous tests. Discussion: https://postgr.es/m/20170503172746.rwftidszir67sgk7@alvherre.pgsql
2017-05-03 20:52:00 +02:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'CREATE SEQUENCE test_table_col1_seq' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE SEQUENCE dump_test.test_table_col1_seq\E
Add CREATE SEQUENCE AS <data type> clause This stores a data type, required to be an integer type, with the sequence. The sequences min and max values default to the range supported by the type, and they cannot be set to values exceeding that range. The internal implementation of the sequence is not affected. Change the serial types to create sequences of the appropriate type. This makes sure that the min and max values of the sequence for a serial column match the range of values supported by the table column. So the sequence can no longer overflow the table column. This also makes monitoring for sequence exhaustion/wraparound easier, which currently requires various contortions to cross-reference the sequences with the table columns they are used with. This commit also effectively reverts the pg_sequence column reordering in f3b421da5f4addc95812b9db05a24972b8fd9739, because the new seqtypid column allows us to fill the hole in the struct and create a more natural overall column ordering. Reviewed-by: Steve Singer <steve@ssinger.info> Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-02-10 21:12:32 +01:00
\n\s+\QAS integer\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n\s+\QSTART WITH 1\E
\n\s+\QINCREMENT BY 1\E
\n\s+\QNO MINVALUE\E
\n\s+\QNO MAXVALUE\E
\n\s+\QCACHE 1;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Local partitioned indexes When CREATE INDEX is run on a partitioned table, create catalog entries for an index on the partitioned table (which is just a placeholder since the table proper has no data of its own), and recurse to create actual indexes on the existing partitions; create them in future partitions also. As a convenience gadget, if the new index definition matches some existing index in partitions, these are picked up and used instead of creating new ones. Whichever way these indexes come about, they become attached to the index on the parent table and are dropped alongside it, and cannot be dropped on isolation unless they are detached first. To support pg_dump'ing these indexes, add commands CREATE INDEX ON ONLY <table> (which creates the index on the parent partitioned table, without recursing) and ALTER INDEX ATTACH PARTITION (which is used after the indexes have been created individually on each partition, to attach them to the parent index). These reconstruct prior database state exactly. Reviewed-by: (in alphabetical order) Peter Eisentraut, Robert Haas, Amit Langote, Jesper Pedersen, Simon Riggs, David Rowley Discussion: https://postgr.es/m/20171113170646.gzweigyrgg6pwsg4@alvherre.pgsql
2018-01-19 15:49:22 +01:00
'CREATE INDEX ON ONLY measurement' => {
create_order => 92,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
create_sql =>
'CREATE INDEX ON dump_test.measurement (city_id, logdate);',
Local partitioned indexes When CREATE INDEX is run on a partitioned table, create catalog entries for an index on the partitioned table (which is just a placeholder since the table proper has no data of its own), and recurse to create actual indexes on the existing partitions; create them in future partitions also. As a convenience gadget, if the new index definition matches some existing index in partitions, these are picked up and used instead of creating new ones. Whichever way these indexes come about, they become attached to the index on the parent table and are dropped alongside it, and cannot be dropped on isolation unless they are detached first. To support pg_dump'ing these indexes, add commands CREATE INDEX ON ONLY <table> (which creates the index on the parent partitioned table, without recursing) and ALTER INDEX ATTACH PARTITION (which is used after the indexes have been created individually on each partition, to attach them to the parent index). These reconstruct prior database state exactly. Reviewed-by: (in alphabetical order) Peter Eisentraut, Robert Haas, Amit Langote, Jesper Pedersen, Simon Riggs, David Rowley Discussion: https://postgr.es/m/20171113170646.gzweigyrgg6pwsg4@alvherre.pgsql
2018-01-19 15:49:22 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE INDEX measurement_city_id_logdate_idx ON ONLY dump_test.measurement USING\E
Local partitioned indexes When CREATE INDEX is run on a partitioned table, create catalog entries for an index on the partitioned table (which is just a placeholder since the table proper has no data of its own), and recurse to create actual indexes on the existing partitions; create them in future partitions also. As a convenience gadget, if the new index definition matches some existing index in partitions, these are picked up and used instead of creating new ones. Whichever way these indexes come about, they become attached to the index on the parent table and are dropped alongside it, and cannot be dropped on isolation unless they are detached first. To support pg_dump'ing these indexes, add commands CREATE INDEX ON ONLY <table> (which creates the index on the parent partitioned table, without recursing) and ALTER INDEX ATTACH PARTITION (which is used after the indexes have been created individually on each partition, to attach them to the parent index). These reconstruct prior database state exactly. Reviewed-by: (in alphabetical order) Peter Eisentraut, Robert Haas, Amit Langote, Jesper Pedersen, Simon Riggs, David Rowley Discussion: https://postgr.es/m/20171113170646.gzweigyrgg6pwsg4@alvherre.pgsql
2018-01-19 15:49:22 +01:00
/xm,
like => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
defaults => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
no_blobs => 1,
no_privs => 1,
no_owner => 1,
only_dump_test_schema => 1,
pg_dumpall_dbprivs => 1,
Add --exclude-database option to pg_dumpall This option functions similarly to pg_dump's --exclude-table option, but for database names. The option can be given once, and the argument can be a pattern including wildcard characters. Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier Discussion: https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
2019-03-01 16:47:44 +01:00
pg_dumpall_exclude => 1,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
schema_only => 1,
section_post_data => 1,
test_schema_plus_blobs => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
},
Local partitioned indexes When CREATE INDEX is run on a partitioned table, create catalog entries for an index on the partitioned table (which is just a placeholder since the table proper has no data of its own), and recurse to create actual indexes on the existing partitions; create them in future partitions also. As a convenience gadget, if the new index definition matches some existing index in partitions, these are picked up and used instead of creating new ones. Whichever way these indexes come about, they become attached to the index on the parent table and are dropped alongside it, and cannot be dropped on isolation unless they are detached first. To support pg_dump'ing these indexes, add commands CREATE INDEX ON ONLY <table> (which creates the index on the parent partitioned table, without recursing) and ALTER INDEX ATTACH PARTITION (which is used after the indexes have been created individually on each partition, to attach them to the parent index). These reconstruct prior database state exactly. Reviewed-by: (in alphabetical order) Peter Eisentraut, Robert Haas, Amit Langote, Jesper Pedersen, Simon Riggs, David Rowley Discussion: https://postgr.es/m/20171113170646.gzweigyrgg6pwsg4@alvherre.pgsql
2018-01-19 15:49:22 +01:00
unlike => {
exclude_dump_test_schema => 1,
only_dump_test_table => 1,
pg_dumpall_globals => 1,
pg_dumpall_globals_clean => 1,
role => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
},
Local partitioned indexes When CREATE INDEX is run on a partitioned table, create catalog entries for an index on the partitioned table (which is just a placeholder since the table proper has no data of its own), and recurse to create actual indexes on the existing partitions; create them in future partitions also. As a convenience gadget, if the new index definition matches some existing index in partitions, these are picked up and used instead of creating new ones. Whichever way these indexes come about, they become attached to the index on the parent table and are dropped alongside it, and cannot be dropped on isolation unless they are detached first. To support pg_dump'ing these indexes, add commands CREATE INDEX ON ONLY <table> (which creates the index on the parent partitioned table, without recursing) and ALTER INDEX ATTACH PARTITION (which is used after the indexes have been created individually on each partition, to attach them to the parent index). These reconstruct prior database state exactly. Reviewed-by: (in alphabetical order) Peter Eisentraut, Robert Haas, Amit Langote, Jesper Pedersen, Simon Riggs, David Rowley Discussion: https://postgr.es/m/20171113170646.gzweigyrgg6pwsg4@alvherre.pgsql
2018-01-19 15:49:22 +01:00
Allow UNIQUE indexes on partitioned tables If we restrict unique constraints on partitioned tables so that they must always include the partition key, then our standard approach to unique indexes already works --- each unique key is forced to exist within a single partition, so enforcing the unique restriction in each index individually is enough to have it enforced globally. Therefore we can implement unique indexes on partitions by simply removing a few restrictions (and adding others.) Discussion: https://postgr.es/m/20171222212921.hi6hg6pem2w2t36z@alvherre.pgsql Discussion: https://postgr.es/m/20171229230607.3iib6b62fn3uaf47@alvherre.pgsql Reviewed-by: Simon Riggs, Jesper Pedersen, Peter Eisentraut, Jaime Casanova, Amit Langote
2018-02-19 20:59:37 +01:00
'ALTER TABLE measurement PRIMARY KEY' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
create_order => 93,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'ALTER TABLE dump_test.measurement ADD PRIMARY KEY (city_id, logdate);',
Allow UNIQUE indexes on partitioned tables If we restrict unique constraints on partitioned tables so that they must always include the partition key, then our standard approach to unique indexes already works --- each unique key is forced to exist within a single partition, so enforcing the unique restriction in each index individually is enough to have it enforced globally. Therefore we can implement unique indexes on partitions by simply removing a few restrictions (and adding others.) Discussion: https://postgr.es/m/20171222212921.hi6hg6pem2w2t36z@alvherre.pgsql Discussion: https://postgr.es/m/20171229230607.3iib6b62fn3uaf47@alvherre.pgsql Reviewed-by: Simon Riggs, Jesper Pedersen, Peter Eisentraut, Jaime Casanova, Amit Langote
2018-02-19 20:59:37 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE ONLY dump_test.measurement\E \n^\s+
Allow UNIQUE indexes on partitioned tables If we restrict unique constraints on partitioned tables so that they must always include the partition key, then our standard approach to unique indexes already works --- each unique key is forced to exist within a single partition, so enforcing the unique restriction in each index individually is enough to have it enforced globally. Therefore we can implement unique indexes on partitions by simply removing a few restrictions (and adding others.) Discussion: https://postgr.es/m/20171222212921.hi6hg6pem2w2t36z@alvherre.pgsql Discussion: https://postgr.es/m/20171229230607.3iib6b62fn3uaf47@alvherre.pgsql Reviewed-by: Simon Riggs, Jesper Pedersen, Peter Eisentraut, Jaime Casanova, Amit Langote
2018-02-19 20:59:37 +01:00
\QADD CONSTRAINT measurement_pkey PRIMARY KEY (city_id, logdate);\E
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Allow UNIQUE indexes on partitioned tables If we restrict unique constraints on partitioned tables so that they must always include the partition key, then our standard approach to unique indexes already works --- each unique key is forced to exist within a single partition, so enforcing the unique restriction in each index individually is enough to have it enforced globally. Therefore we can implement unique indexes on partitions by simply removing a few restrictions (and adding others.) Discussion: https://postgr.es/m/20171222212921.hi6hg6pem2w2t36z@alvherre.pgsql Discussion: https://postgr.es/m/20171229230607.3iib6b62fn3uaf47@alvherre.pgsql Reviewed-by: Simon Riggs, Jesper Pedersen, Peter Eisentraut, Jaime Casanova, Amit Langote
2018-02-19 20:59:37 +01:00
Local partitioned indexes When CREATE INDEX is run on a partitioned table, create catalog entries for an index on the partitioned table (which is just a placeholder since the table proper has no data of its own), and recurse to create actual indexes on the existing partitions; create them in future partitions also. As a convenience gadget, if the new index definition matches some existing index in partitions, these are picked up and used instead of creating new ones. Whichever way these indexes come about, they become attached to the index on the parent table and are dropped alongside it, and cannot be dropped on isolation unless they are detached first. To support pg_dump'ing these indexes, add commands CREATE INDEX ON ONLY <table> (which creates the index on the parent partitioned table, without recursing) and ALTER INDEX ATTACH PARTITION (which is used after the indexes have been created individually on each partition, to attach them to the parent index). These reconstruct prior database state exactly. Reviewed-by: (in alphabetical order) Peter Eisentraut, Robert Haas, Amit Langote, Jesper Pedersen, Simon Riggs, David Rowley Discussion: https://postgr.es/m/20171113170646.gzweigyrgg6pwsg4@alvherre.pgsql
2018-01-19 15:49:22 +01:00
'CREATE INDEX ... ON measurement_y2006_m2' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE INDEX measurement_y2006m2_city_id_logdate_idx ON dump_test_second_schema.measurement_y2006m2 \E
Local partitioned indexes When CREATE INDEX is run on a partitioned table, create catalog entries for an index on the partitioned table (which is just a placeholder since the table proper has no data of its own), and recurse to create actual indexes on the existing partitions; create them in future partitions also. As a convenience gadget, if the new index definition matches some existing index in partitions, these are picked up and used instead of creating new ones. Whichever way these indexes come about, they become attached to the index on the parent table and are dropped alongside it, and cannot be dropped on isolation unless they are detached first. To support pg_dump'ing these indexes, add commands CREATE INDEX ON ONLY <table> (which creates the index on the parent partitioned table, without recursing) and ALTER INDEX ATTACH PARTITION (which is used after the indexes have been created individually on each partition, to attach them to the parent index). These reconstruct prior database state exactly. Reviewed-by: (in alphabetical order) Peter Eisentraut, Robert Haas, Amit Langote, Jesper Pedersen, Simon Riggs, David Rowley Discussion: https://postgr.es/m/20171113170646.gzweigyrgg6pwsg4@alvherre.pgsql
2018-01-19 15:49:22 +01:00
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
role => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
},
Local partitioned indexes When CREATE INDEX is run on a partitioned table, create catalog entries for an index on the partitioned table (which is just a placeholder since the table proper has no data of its own), and recurse to create actual indexes on the existing partitions; create them in future partitions also. As a convenience gadget, if the new index definition matches some existing index in partitions, these are picked up and used instead of creating new ones. Whichever way these indexes come about, they become attached to the index on the parent table and are dropped alongside it, and cannot be dropped on isolation unless they are detached first. To support pg_dump'ing these indexes, add commands CREATE INDEX ON ONLY <table> (which creates the index on the parent partitioned table, without recursing) and ALTER INDEX ATTACH PARTITION (which is used after the indexes have been created individually on each partition, to attach them to the parent index). These reconstruct prior database state exactly. Reviewed-by: (in alphabetical order) Peter Eisentraut, Robert Haas, Amit Langote, Jesper Pedersen, Simon Riggs, David Rowley Discussion: https://postgr.es/m/20171113170646.gzweigyrgg6pwsg4@alvherre.pgsql
2018-01-19 15:49:22 +01:00
'ALTER INDEX ... ATTACH PARTITION' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Local partitioned indexes When CREATE INDEX is run on a partitioned table, create catalog entries for an index on the partitioned table (which is just a placeholder since the table proper has no data of its own), and recurse to create actual indexes on the existing partitions; create them in future partitions also. As a convenience gadget, if the new index definition matches some existing index in partitions, these are picked up and used instead of creating new ones. Whichever way these indexes come about, they become attached to the index on the parent table and are dropped alongside it, and cannot be dropped on isolation unless they are detached first. To support pg_dump'ing these indexes, add commands CREATE INDEX ON ONLY <table> (which creates the index on the parent partitioned table, without recursing) and ALTER INDEX ATTACH PARTITION (which is used after the indexes have been created individually on each partition, to attach them to the parent index). These reconstruct prior database state exactly. Reviewed-by: (in alphabetical order) Peter Eisentraut, Robert Haas, Amit Langote, Jesper Pedersen, Simon Riggs, David Rowley Discussion: https://postgr.es/m/20171113170646.gzweigyrgg6pwsg4@alvherre.pgsql
2018-01-19 15:49:22 +01:00
\QALTER INDEX dump_test.measurement_city_id_logdate_idx ATTACH PARTITION dump_test_second_schema.measurement_y2006m2_city_id_logdate_idx\E
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
role => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_post_data => 1,
},
},
Allow UNIQUE indexes on partitioned tables If we restrict unique constraints on partitioned tables so that they must always include the partition key, then our standard approach to unique indexes already works --- each unique key is forced to exist within a single partition, so enforcing the unique restriction in each index individually is enough to have it enforced globally. Therefore we can implement unique indexes on partitions by simply removing a few restrictions (and adding others.) Discussion: https://postgr.es/m/20171222212921.hi6hg6pem2w2t36z@alvherre.pgsql Discussion: https://postgr.es/m/20171229230607.3iib6b62fn3uaf47@alvherre.pgsql Reviewed-by: Simon Riggs, Jesper Pedersen, Peter Eisentraut, Jaime Casanova, Amit Langote
2018-02-19 20:59:37 +01:00
'ALTER INDEX ... ATTACH PARTITION (primary key)' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
all_runs => 1,
catch_all => 'CREATE ... commands',
regexp => qr/^
Allow UNIQUE indexes on partitioned tables If we restrict unique constraints on partitioned tables so that they must always include the partition key, then our standard approach to unique indexes already works --- each unique key is forced to exist within a single partition, so enforcing the unique restriction in each index individually is enough to have it enforced globally. Therefore we can implement unique indexes on partitions by simply removing a few restrictions (and adding others.) Discussion: https://postgr.es/m/20171222212921.hi6hg6pem2w2t36z@alvherre.pgsql Discussion: https://postgr.es/m/20171229230607.3iib6b62fn3uaf47@alvherre.pgsql Reviewed-by: Simon Riggs, Jesper Pedersen, Peter Eisentraut, Jaime Casanova, Amit Langote
2018-02-19 20:59:37 +01:00
\QALTER INDEX dump_test.measurement_pkey ATTACH PARTITION dump_test_second_schema.measurement_y2006m2_pkey\E
/xm,
like => {
binary_upgrade => 1,
clean => 1,
clean_if_exists => 1,
createdb => 1,
defaults => 1,
exclude_dump_test_schema => 1,
exclude_test_table => 1,
exclude_test_table_data => 1,
no_blobs => 1,
no_privs => 1,
no_owner => 1,
pg_dumpall_dbprivs => 1,
Add --exclude-database option to pg_dumpall This option functions similarly to pg_dump's --exclude-table option, but for database names. The option can be given once, and the argument can be a pattern including wildcard characters. Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier Discussion: https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
2019-03-01 16:47:44 +01:00
pg_dumpall_exclude => 1,
Allow UNIQUE indexes on partitioned tables If we restrict unique constraints on partitioned tables so that they must always include the partition key, then our standard approach to unique indexes already works --- each unique key is forced to exist within a single partition, so enforcing the unique restriction in each index individually is enough to have it enforced globally. Therefore we can implement unique indexes on partitions by simply removing a few restrictions (and adding others.) Discussion: https://postgr.es/m/20171222212921.hi6hg6pem2w2t36z@alvherre.pgsql Discussion: https://postgr.es/m/20171229230607.3iib6b62fn3uaf47@alvherre.pgsql Reviewed-by: Simon Riggs, Jesper Pedersen, Peter Eisentraut, Jaime Casanova, Amit Langote
2018-02-19 20:59:37 +01:00
role => 1,
schema_only => 1,
section_post_data => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
},
Allow UNIQUE indexes on partitioned tables If we restrict unique constraints on partitioned tables so that they must always include the partition key, then our standard approach to unique indexes already works --- each unique key is forced to exist within a single partition, so enforcing the unique restriction in each index individually is enough to have it enforced globally. Therefore we can implement unique indexes on partitions by simply removing a few restrictions (and adding others.) Discussion: https://postgr.es/m/20171222212921.hi6hg6pem2w2t36z@alvherre.pgsql Discussion: https://postgr.es/m/20171229230607.3iib6b62fn3uaf47@alvherre.pgsql Reviewed-by: Simon Riggs, Jesper Pedersen, Peter Eisentraut, Jaime Casanova, Amit Langote
2018-02-19 20:59:37 +01:00
unlike => {
only_dump_test_schema => 1,
only_dump_test_table => 1,
pg_dumpall_globals => 1,
Do not DROP default roles in pg_dumpall -c When pulling the list of roles to drop, exclude roles whose names begin with "pg_" (as we do when we are dumping the roles out to recreate them). Also add regression tests to cover pg_dumpall -c and this specific issue. Noticed by Rushabh Lathia. Patch by me.
2016-05-25 05:31:55 +02:00
pg_dumpall_globals_clean => 1,
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
section_pre_data => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
test_schema_plus_blobs => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'CREATE VIEW test_view' => {
create_order => 61,
create_sql => 'CREATE VIEW dump_test.test_view
WITH (check_option = \'local\', security_barrier = true) AS
SELECT col1 FROM dump_test.test_table;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QCREATE VIEW dump_test.test_view WITH (security_barrier='true') AS\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n\s+\QSELECT test_table.col1\E
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\n\s+\QFROM dump_test.test_table\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
\n\s+\QWITH LOCAL CHECK OPTION;\E/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'ALTER VIEW test_view SET DEFAULT' => {
create_order => 62,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
'ALTER VIEW dump_test.test_view ALTER COLUMN col1 SET DEFAULT 1;',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QALTER TABLE ONLY dump_test.test_view ALTER COLUMN col1 SET DEFAULT 1;\E/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { exclude_dump_test_schema => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# FIXME
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
'DROP SCHEMA public (for testing without public schema)' => {
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
database => 'regress_pg_dump_test',
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
create_order => 100,
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
create_sql => 'DROP SCHEMA public;',
regexp => qr/^DROP SCHEMA public;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
'DROP SCHEMA public' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP SCHEMA public;/m,
Improve pg_dump's handling of "special" built-in objects. We had some pretty ad-hoc handling of the public schema and the plpgsql extension, which are both presumed to exist in template0 but might be modified or deleted in the database being dumped. Up to now, by default pg_dump would emit a CREATE EXTENSION IF NOT EXISTS command as well as a COMMENT command for plpgsql. The usefulness of the former is questionable, and the latter caused annoying errors in non-superuser dump/restore scenarios. Let's instead install a rule that built-in extensions (identified by having low-numbered OIDs) are not to be dumped. We were doing it that way already in binary-upgrade mode, so this just makes regular mode behave the same. It remains true that if someone has installed a non-default ACL on the plpgsql language, that will get dumped thanks to the pg_init_privs mechanism. This is more consistent with the handling of built-in objects of other kinds. Also, change the very ad-hoc mechanism that was used to avoid dumping creation and comment commands for the public schema. Instead of hardwiring a test in _printTocEntry(), make use of the DUMP_COMPONENT_ infrastructure to mark that schema up-front about what we want to do with it. This has the visible effect that the public schema won't be mentioned in the output at all, except for updating its ACL if it has a non-default ACL. Previously, while it was normally not mentioned, --clean mode would drop and recreate it, again causing headaches for non-superuser usage. This change likewise makes the public schema less special and more like other built-in objects. If plpgsql, or the public schema, has been removed entirely in the source DB, that situation won't be reproduced in the destination ... but that was true before. Discussion: https://postgr.es/m/29048.1516812451@sss.pgh.pa.us
2018-01-25 19:54:42 +01:00
# this shouldn't ever get emitted anymore
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
'DROP SCHEMA IF EXISTS public' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP SCHEMA IF EXISTS public;/m,
Improve pg_dump's handling of "special" built-in objects. We had some pretty ad-hoc handling of the public schema and the plpgsql extension, which are both presumed to exist in template0 but might be modified or deleted in the database being dumped. Up to now, by default pg_dump would emit a CREATE EXTENSION IF NOT EXISTS command as well as a COMMENT command for plpgsql. The usefulness of the former is questionable, and the latter caused annoying errors in non-superuser dump/restore scenarios. Let's instead install a rule that built-in extensions (identified by having low-numbered OIDs) are not to be dumped. We were doing it that way already in binary-upgrade mode, so this just makes regular mode behave the same. It remains true that if someone has installed a non-default ACL on the plpgsql language, that will get dumped thanks to the pg_init_privs mechanism. This is more consistent with the handling of built-in objects of other kinds. Also, change the very ad-hoc mechanism that was used to avoid dumping creation and comment commands for the public schema. Instead of hardwiring a test in _printTocEntry(), make use of the DUMP_COMPONENT_ infrastructure to mark that schema up-front about what we want to do with it. This has the visible effect that the public schema won't be mentioned in the output at all, except for updating its ACL if it has a non-default ACL. Previously, while it was normally not mentioned, --clean mode would drop and recreate it, again causing headaches for non-superuser usage. This change likewise makes the public schema less special and more like other built-in objects. If plpgsql, or the public schema, has been removed entirely in the source DB, that situation won't be reproduced in the destination ... but that was true before. Discussion: https://postgr.es/m/29048.1516812451@sss.pgh.pa.us
2018-01-25 19:54:42 +01:00
# this shouldn't ever get emitted anymore
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP EXTENSION plpgsql' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP EXTENSION plpgsql;/m,
Improve pg_dump's handling of "special" built-in objects. We had some pretty ad-hoc handling of the public schema and the plpgsql extension, which are both presumed to exist in template0 but might be modified or deleted in the database being dumped. Up to now, by default pg_dump would emit a CREATE EXTENSION IF NOT EXISTS command as well as a COMMENT command for plpgsql. The usefulness of the former is questionable, and the latter caused annoying errors in non-superuser dump/restore scenarios. Let's instead install a rule that built-in extensions (identified by having low-numbered OIDs) are not to be dumped. We were doing it that way already in binary-upgrade mode, so this just makes regular mode behave the same. It remains true that if someone has installed a non-default ACL on the plpgsql language, that will get dumped thanks to the pg_init_privs mechanism. This is more consistent with the handling of built-in objects of other kinds. Also, change the very ad-hoc mechanism that was used to avoid dumping creation and comment commands for the public schema. Instead of hardwiring a test in _printTocEntry(), make use of the DUMP_COMPONENT_ infrastructure to mark that schema up-front about what we want to do with it. This has the visible effect that the public schema won't be mentioned in the output at all, except for updating its ACL if it has a non-default ACL. Previously, while it was normally not mentioned, --clean mode would drop and recreate it, again causing headaches for non-superuser usage. This change likewise makes the public schema less special and more like other built-in objects. If plpgsql, or the public schema, has been removed entirely in the source DB, that situation won't be reproduced in the destination ... but that was true before. Discussion: https://postgr.es/m/29048.1516812451@sss.pgh.pa.us
2018-01-25 19:54:42 +01:00
# this shouldn't ever get emitted anymore
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP FUNCTION dump_test.pltestlang_call_handler()' => {
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
regexp => qr/^DROP FUNCTION dump_test\.pltestlang_call_handler\(\);/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP LANGUAGE pltestlang' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP PROCEDURAL LANGUAGE pltestlang;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP SCHEMA dump_test' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP SCHEMA dump_test;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP SCHEMA dump_test_second_schema' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP SCHEMA dump_test_second_schema;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP TABLE test_table' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP TABLE dump_test\.test_table;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP TABLE fk_reference_test_table' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP TABLE dump_test\.fk_reference_test_table;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP TABLE test_second_table' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP TABLE dump_test\.test_second_table;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP EXTENSION IF EXISTS plpgsql' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP EXTENSION IF EXISTS plpgsql;/m,
Improve pg_dump's handling of "special" built-in objects. We had some pretty ad-hoc handling of the public schema and the plpgsql extension, which are both presumed to exist in template0 but might be modified or deleted in the database being dumped. Up to now, by default pg_dump would emit a CREATE EXTENSION IF NOT EXISTS command as well as a COMMENT command for plpgsql. The usefulness of the former is questionable, and the latter caused annoying errors in non-superuser dump/restore scenarios. Let's instead install a rule that built-in extensions (identified by having low-numbered OIDs) are not to be dumped. We were doing it that way already in binary-upgrade mode, so this just makes regular mode behave the same. It remains true that if someone has installed a non-default ACL on the plpgsql language, that will get dumped thanks to the pg_init_privs mechanism. This is more consistent with the handling of built-in objects of other kinds. Also, change the very ad-hoc mechanism that was used to avoid dumping creation and comment commands for the public schema. Instead of hardwiring a test in _printTocEntry(), make use of the DUMP_COMPONENT_ infrastructure to mark that schema up-front about what we want to do with it. This has the visible effect that the public schema won't be mentioned in the output at all, except for updating its ACL if it has a non-default ACL. Previously, while it was normally not mentioned, --clean mode would drop and recreate it, again causing headaches for non-superuser usage. This change likewise makes the public schema less special and more like other built-in objects. If plpgsql, or the public schema, has been removed entirely in the source DB, that situation won't be reproduced in the destination ... but that was true before. Discussion: https://postgr.es/m/29048.1516812451@sss.pgh.pa.us
2018-01-25 19:54:42 +01:00
# this shouldn't ever get emitted anymore
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP FUNCTION IF EXISTS dump_test.pltestlang_call_handler()' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\QDROP FUNCTION IF EXISTS dump_test.pltestlang_call_handler();\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean_if_exists => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP LANGUAGE IF EXISTS pltestlang' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP PROCEDURAL LANGUAGE IF EXISTS pltestlang;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean_if_exists => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP SCHEMA IF EXISTS dump_test' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP SCHEMA IF EXISTS dump_test;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean_if_exists => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP SCHEMA IF EXISTS dump_test_second_schema' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP SCHEMA IF EXISTS dump_test_second_schema;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean_if_exists => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP TABLE IF EXISTS test_table' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP TABLE IF EXISTS dump_test\.test_table;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean_if_exists => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'DROP TABLE IF EXISTS test_second_table' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^DROP TABLE IF EXISTS dump_test\.test_second_table;/m,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { clean_if_exists => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
'DROP ROLE regress_dump_test_role' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
\QDROP ROLE regress_dump_test_role;\E
Do not DROP default roles in pg_dumpall -c When pulling the list of roles to drop, exclude roles whose names begin with "pg_" (as we do when we are dumping the roles out to recreate them). Also add regression tests to cover pg_dumpall -c and this specific issue. Noticed by Rushabh Lathia. Patch by me.
2016-05-25 05:31:55 +02:00
/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { pg_dumpall_globals_clean => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Do not DROP default roles in pg_dumpall -c When pulling the list of roles to drop, exclude roles whose names begin with "pg_" (as we do when we are dumping the roles out to recreate them). Also add regression tests to cover pg_dumpall -c and this specific issue. Noticed by Rushabh Lathia. Patch by me.
2016-05-25 05:31:55 +02:00
'DROP ROLE pg_' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Align better test output regex with grammar in pg_dump TAP tests This enforces one-or-more character matches in the regular expressions for pg_dump testing on SQL syntax output where zero-or-more matches implies a syntax error. Author: Daniel Gustafsson Reviewed-by: David G. Johnston, Michael Paquier Discussion: https://postgr.es/m/B313C32C-0E24-4AFB-95FF-6DA0C4E18A89@yesql.se
2019-02-07 02:04:55 +01:00
\QDROP ROLE pg_\E.+;
Do not DROP default roles in pg_dumpall -c When pulling the list of roles to drop, exclude roles whose names begin with "pg_" (as we do when we are dumping the roles out to recreate them). Also add regression tests to cover pg_dumpall -c and this specific issue. Noticed by Rushabh Lathia. Patch by me.
2016-05-25 05:31:55 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# this shouldn't ever get emitted anywhere
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'GRANT USAGE ON SCHEMA dump_test_second_schema' => {
create_order => 10,
create_sql => 'GRANT USAGE ON SCHEMA dump_test_second_schema
TO regress_dump_test_role;',
regexp => qr/^
\QGRANT USAGE ON SCHEMA dump_test_second_schema TO regress_dump_test_role;\E
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
role => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
unlike => { no_privs => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'GRANT USAGE ON FOREIGN DATA WRAPPER dummy' => {
create_order => 85,
create_sql => 'GRANT USAGE ON FOREIGN DATA WRAPPER dummy
TO regress_dump_test_role;',
regexp => qr/^
\QGRANT ALL ON FOREIGN DATA WRAPPER dummy TO regress_dump_test_role;\E
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_privs => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'GRANT USAGE ON FOREIGN SERVER s1' => {
create_order => 85,
create_sql => 'GRANT USAGE ON FOREIGN SERVER s1
TO regress_dump_test_role;',
regexp => qr/^
\QGRANT ALL ON FOREIGN SERVER s1 TO regress_dump_test_role;\E
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_privs => 1, },
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'GRANT USAGE ON DOMAIN dump_test.us_postal_code' => {
create_order => 72,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'GRANT USAGE ON DOMAIN dump_test.us_postal_code TO regress_dump_test_role;',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QGRANT ALL ON TYPE dump_test.us_postal_code TO regress_dump_test_role;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_privs => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'GRANT USAGE ON TYPE dump_test.int42' => {
create_order => 87,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
'GRANT USAGE ON TYPE dump_test.int42 TO regress_dump_test_role;',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QGRANT ALL ON TYPE dump_test.int42 TO regress_dump_test_role;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_privs => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'GRANT USAGE ON TYPE dump_test.planets - ENUM' => {
create_order => 66,
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
create_sql =>
'GRANT USAGE ON TYPE dump_test.planets TO regress_dump_test_role;',
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QGRANT ALL ON TYPE dump_test.planets TO regress_dump_test_role;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_privs => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'GRANT USAGE ON TYPE dump_test.textrange - RANGE' => {
create_order => 67,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
create_sql =>
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'GRANT USAGE ON TYPE dump_test.textrange TO regress_dump_test_role;',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QGRANT ALL ON TYPE dump_test.textrange TO regress_dump_test_role;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_privs => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Correctly dump database and tablespace ACLs Dump out the appropriate GRANT/REVOKE commands for databases and tablespaces from pg_dumpall to replicate what the current state is. This was broken during the changes to buildACLCommands for 9.6+ servers for pg_init_privs.
2016-07-17 15:04:46 +02:00
'GRANT CREATE ON DATABASE dump_test' => {
create_order => 48,
Final pgindent + perltidy run for 9.6.
2016-08-15 19:42:51 +02:00
create_sql =>
'GRANT CREATE ON DATABASE dump_test TO regress_dump_test_role;',
Correctly dump database and tablespace ACLs Dump out the appropriate GRANT/REVOKE commands for databases and tablespaces from pg_dumpall to replicate what the current state is. This was broken during the changes to buildACLCommands for 9.6+ servers for pg_init_privs.
2016-07-17 15:04:46 +02:00
regexp => qr/^
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
\QGRANT CREATE ON DATABASE dump_test TO regress_dump_test_role;\E
Correctly dump database and tablespace ACLs Dump out the appropriate GRANT/REVOKE commands for databases and tablespaces from pg_dumpall to replicate what the current state is. This was broken during the changes to buildACLCommands for 9.6+ servers for pg_init_privs.
2016-07-17 15:04:46 +02:00
/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { pg_dumpall_dbprivs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'GRANT SELECT ON TABLE test_table' => {
create_order => 5,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'GRANT SELECT ON TABLE dump_test.test_table
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
TO regress_dump_test_role;',
Final pgindent + perltidy run for 9.6.
2016-08-15 19:42:51 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QGRANT SELECT ON TABLE dump_test.test_table TO regress_dump_test_role;\E/m,
Final pgindent + perltidy run for 9.6.
2016-08-15 19:42:51 +02:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
exclude_dump_test_schema => 1,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
exclude_test_table => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_privs => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'GRANT SELECT ON TABLE measurement' => {
create_order => 91,
create_sql => 'GRANT SELECT ON
TABLE dump_test.measurement
TO regress_dump_test_role;',
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QGRANT SELECT ON TABLE dump_test.measurement TO regress_dump_test_role;\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_privs => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'GRANT SELECT ON TABLE measurement_y2006m2' => {
create_order => 92,
create_sql => 'GRANT SELECT ON
TABLE dump_test_second_schema.measurement_y2006m2
TO regress_dump_test_role;',
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QGRANT SELECT ON TABLE dump_test_second_schema.measurement_y2006m2 TO regress_dump_test_role;\E/m,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
role => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
unlike => { no_privs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'GRANT ALL ON LARGE OBJECT ...' => {
create_order => 60,
create_sql => 'DO $$
DECLARE myoid oid;
BEGIN
SELECT loid FROM pg_largeobject INTO myoid;
EXECUTE \'GRANT ALL ON LARGE OBJECT \' || myoid || \' TO regress_dump_test_role;\';
END;
$$;',
regexp => qr/^
\QGRANT ALL ON LARGE OBJECT \E[0-9]+\Q TO regress_dump_test_role;\E
/xm,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
column_inserts => 1,
data_only => 1,
section_pre_data => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
test_schema_plus_blobs => 1,
Fix pg_upgrade for oid removal. pg_upgrade previously copied pg_largeobject_metadata over from the old cluster. That doesn't work, because the table has oids before 578b229718. I missed that. As most pieces of metadata for large objects already were dumped as DDL (except for comments overwritten by pg_upgrade, due to the copy of pg_largeobject_metadata) it seems reasonable to just also dump grants for large objects. If we ever consider this a relevant performance problem, we'd need to fix the rest of the already emitted DDL too. There's still an open discussion about whether we'll want to force a specific ordering for the dumped objects, as currently pg_largeobjects_metadata potentially has a different ordering before/after pg_upgrade, which can make automated testing a bit harder. Reported-By: Andrew Dunstan Author: Andres Freund Discussion: https://postgr.es/m/91a8a980-41bc-412b-fba2-2ba71a141c2b@2ndQuadrant.com
2018-11-26 23:20:36 +01:00
binary_upgrade => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
no_blobs => 1,
no_privs => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'GRANT INSERT(col1) ON TABLE test_second_table' => {
create_order => 8,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql =>
'GRANT INSERT (col1) ON TABLE dump_test.test_second_table
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
TO regress_dump_test_role;',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QGRANT INSERT(col1) ON TABLE dump_test.test_second_table TO regress_dump_test_role;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
unlike => {
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
no_privs => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
'GRANT EXECUTE ON FUNCTION pg_sleep() TO regress_dump_test_role' => {
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
create_order => 16,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'GRANT EXECUTE ON FUNCTION pg_sleep(float8)
Establish conventions about global object names used in regression tests. To ensure that "make installcheck" can be used safely against an existing installation, we need to be careful about what global object names (database, role, and tablespace names) we use; otherwise we might accidentally clobber important objects. There's been a weak consensus that test databases should have names including "regression", and that test role names should start with "regress_", but we didn't have any particular rule about tablespace names; and neither of the other rules was followed with any consistency either. This commit moves us a long way towards having a hard-and-fast rule that regression test databases must have names including "regression", and that test role and tablespace names must start with "regress_". It's not completely there because I did not touch some test cases in rolenames.sql that test creation of special role names like "session_user". That will require some rethinking of exactly what we want to test, whereas the intent of this patch is just to hit all the cases in which the needed renamings are cosmetic. There is no enforcement mechanism in this patch either, but if we don't add one we can expect that the tests will soon be violating the convention again. Again, that's not such a cosmetic change and it will require discussion. (But I did use a quick-hack enforcement patch to find these cases.) Discussion: <16638.1468620817@sss.pgh.pa.us>
2016-07-18 00:42:31 +02:00
TO regress_dump_test_role;',
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QGRANT ALL ON FUNCTION pg_catalog.pg_sleep(double precision) TO regress_dump_test_role;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_privs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'GRANT SELECT (proname ...) ON TABLE pg_proc TO public' => {
create_order => 46,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'GRANT SELECT (
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
tableoid,
oid,
proname,
pronamespace,
proowner,
prolang,
procost,
prorows,
provariadic,
Create the infrastructure for planner support functions. Rename/repurpose pg_proc.protransform as "prosupport". The idea is still that it names an internal function that provides knowledge to the planner about the behavior of the function it's attached to; but redesign the API specification so that it's not limited to doing just one thing, but can support an extensible set of requests. The original purpose of simplifying a function call is handled by the first request type to be invented, SupportRequestSimplify. Adjust all the existing transform functions to handle this API, and rename them fron "xxx_transform" to "xxx_support" to reflect the potential generalization of what they do. (Since we never previously provided any way for extensions to add transform functions, this change doesn't create an API break for them.) Also add DDL and pg_dump support for attaching a support function to a user-defined function. Unfortunately, DDL access has to be restricted to superusers, at least for now; but seeing that support functions will pretty much have to be written in C, that limitation is just theoretical. (This support is untested in this patch, but a follow-on patch will add cases that exercise it.) Discussion: https://postgr.es/m/15193.1548028093@sss.pgh.pa.us
2019-02-10 00:08:48 +01:00
prosupport,
Add prokind column, replacing proisagg and proiswindow The new column distinguishes normal functions, procedures, aggregates, and window functions. This replaces the existing columns proisagg and proiswindow, and replaces the convention that procedures are indicated by prorettype == 0. Also change prorettype to be VOIDOID for procedures. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Michael Paquier <michael@paquier.xyz>
2018-03-02 14:57:38 +01:00
prokind,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
prosecdef,
proleakproof,
proisstrict,
proretset,
provolatile,
proparallel,
pronargs,
pronargdefaults,
prorettype,
proargtypes,
proallargtypes,
proargmodes,
proargnames,
proargdefaults,
protrftypes,
prosrc,
probin,
proconfig,
proacl
) ON TABLE pg_proc TO public;',
regexp => qr/
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QGRANT SELECT(tableoid) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(oid) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proname) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(pronamespace) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proowner) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(prolang) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(procost) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(prorows) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(provariadic) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
Create the infrastructure for planner support functions. Rename/repurpose pg_proc.protransform as "prosupport". The idea is still that it names an internal function that provides knowledge to the planner about the behavior of the function it's attached to; but redesign the API specification so that it's not limited to doing just one thing, but can support an extensible set of requests. The original purpose of simplifying a function call is handled by the first request type to be invented, SupportRequestSimplify. Adjust all the existing transform functions to handle this API, and rename them fron "xxx_transform" to "xxx_support" to reflect the potential generalization of what they do. (Since we never previously provided any way for extensions to add transform functions, this change doesn't create an API break for them.) Also add DDL and pg_dump support for attaching a support function to a user-defined function. Unfortunately, DDL access has to be restricted to superusers, at least for now; but seeing that support functions will pretty much have to be written in C, that limitation is just theoretical. (This support is untested in this patch, but a follow-on patch will add cases that exercise it.) Discussion: https://postgr.es/m/15193.1548028093@sss.pgh.pa.us
2019-02-10 00:08:48 +01:00
\QGRANT SELECT(prosupport) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
Add prokind column, replacing proisagg and proiswindow The new column distinguishes normal functions, procedures, aggregates, and window functions. This replaces the existing columns proisagg and proiswindow, and replaces the convention that procedures are indicated by prorettype == 0. Also change prorettype to be VOIDOID for procedures. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Michael Paquier <michael@paquier.xyz>
2018-03-02 14:57:38 +01:00
\QGRANT SELECT(prokind) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QGRANT SELECT(prosecdef) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proleakproof) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proisstrict) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proretset) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(provolatile) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proparallel) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(pronargs) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(pronargdefaults) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(prorettype) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proargtypes) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proallargtypes) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proargmodes) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proargnames) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proargdefaults) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(protrftypes) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(prosrc) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(probin) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proconfig) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E\n.*
\QGRANT SELECT(proacl) ON TABLE pg_catalog.pg_proc TO PUBLIC;\E/xms,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_privs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
pg_dump: Properly handle public schema ACLs with --clean pg_dump has always handled the public schema in a special way when it comes to the "--clean" option. To wit, we do not drop or recreate the public schema in "normal" mode, but when we are run in "--clean" mode then we do drop and recreate the public schema. When running in "--clean" mode, the public schema is dropped and then recreated and it is recreated with the normal schema-default privileges of "nothing". This is unlike how the public schema starts life, which is to have CREATE and USAGE GRANT'd to the PUBLIC role, and that is what is recorded in pg_init_privs. Due to this, in "--clean" mode, pg_dump would mistakenly only dump out the set of privileges required to go from the initdb-time privileges on the public schema to whatever the current-state privileges are. If the privileges were not changed from initdb time, then no privileges would be dumped out for the public schema, but with the schema being dropped and recreated, the result was that the public schema would have no ACLs on it instead of what it should have, which is the initdb-time privileges. Practically speaking, this meant that pg_dump with --clean mode dumping a database where the ACLs on the public schema were not changed from the default would, upon restore, result in a public schema with *no* privileges GRANT'd, not matching the state of the existing database (where the initdb-time privileges would have been CREATE and USAGE to the PUBLIC role for the public schema). To fix, adjust the query in getNamespaces() to ignore the pg_init_privs entry for the public schema when running in "--clean" mode, meaning that the privileges for the public schema would be dumped, correctly, as if it was going from a newly-created schema to the current state (which is, indeed, what will happen during the restore thanks to the DROP/CREATE). Only the public schema is handled in this special way by pg_dump, no other initdb-time objects are dropped/recreated in --clean mode. Back-patch to 9.6 where the bug was introduced. Discussion: https://postgr.es/m/3534542.o3cNaKiDID%40techfox
2017-03-07 05:29:02 +01:00
'GRANT USAGE ON SCHEMA public TO public' => {
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
regexp => qr/^
pg_dump: Properly handle public schema ACLs with --clean pg_dump has always handled the public schema in a special way when it comes to the "--clean" option. To wit, we do not drop or recreate the public schema in "normal" mode, but when we are run in "--clean" mode then we do drop and recreate the public schema. When running in "--clean" mode, the public schema is dropped and then recreated and it is recreated with the normal schema-default privileges of "nothing". This is unlike how the public schema starts life, which is to have CREATE and USAGE GRANT'd to the PUBLIC role, and that is what is recorded in pg_init_privs. Due to this, in "--clean" mode, pg_dump would mistakenly only dump out the set of privileges required to go from the initdb-time privileges on the public schema to whatever the current-state privileges are. If the privileges were not changed from initdb time, then no privileges would be dumped out for the public schema, but with the schema being dropped and recreated, the result was that the public schema would have no ACLs on it instead of what it should have, which is the initdb-time privileges. Practically speaking, this meant that pg_dump with --clean mode dumping a database where the ACLs on the public schema were not changed from the default would, upon restore, result in a public schema with *no* privileges GRANT'd, not matching the state of the existing database (where the initdb-time privileges would have been CREATE and USAGE to the PUBLIC role for the public schema). To fix, adjust the query in getNamespaces() to ignore the pg_init_privs entry for the public schema when running in "--clean" mode, meaning that the privileges for the public schema would be dumped, correctly, as if it was going from a newly-created schema to the current state (which is, indeed, what will happen during the restore thanks to the DROP/CREATE). Only the public schema is handled in this special way by pg_dump, no other initdb-time objects are dropped/recreated in --clean mode. Back-patch to 9.6 where the bug was introduced. Discussion: https://postgr.es/m/3534542.o3cNaKiDID%40techfox
2017-03-07 05:29:02 +01:00
\Q--\E\n\n
\QGRANT USAGE ON SCHEMA public TO PUBLIC;\E
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
Improve pg_dump's handling of "special" built-in objects. We had some pretty ad-hoc handling of the public schema and the plpgsql extension, which are both presumed to exist in template0 but might be modified or deleted in the database being dumped. Up to now, by default pg_dump would emit a CREATE EXTENSION IF NOT EXISTS command as well as a COMMENT command for plpgsql. The usefulness of the former is questionable, and the latter caused annoying errors in non-superuser dump/restore scenarios. Let's instead install a rule that built-in extensions (identified by having low-numbered OIDs) are not to be dumped. We were doing it that way already in binary-upgrade mode, so this just makes regular mode behave the same. It remains true that if someone has installed a non-default ACL on the plpgsql language, that will get dumped thanks to the pg_init_privs mechanism. This is more consistent with the handling of built-in objects of other kinds. Also, change the very ad-hoc mechanism that was used to avoid dumping creation and comment commands for the public schema. Instead of hardwiring a test in _printTocEntry(), make use of the DUMP_COMPONENT_ infrastructure to mark that schema up-front about what we want to do with it. This has the visible effect that the public schema won't be mentioned in the output at all, except for updating its ACL if it has a non-default ACL. Previously, while it was normally not mentioned, --clean mode would drop and recreate it, again causing headaches for non-superuser usage. This change likewise makes the public schema less special and more like other built-in objects. If plpgsql, or the public schema, has been removed entirely in the source DB, that situation won't be reproduced in the destination ... but that was true before. Discussion: https://postgr.es/m/29048.1516812451@sss.pgh.pa.us
2018-01-25 19:54:42 +01:00
# this shouldn't ever get emitted anymore
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'REFRESH MATERIALIZED VIEW matview' => {
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
regexp => qr/^\QREFRESH MATERIALIZED VIEW dump_test.matview;\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'REFRESH MATERIALIZED VIEW matview_second' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QREFRESH MATERIALIZED VIEW dump_test.matview;\E
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\n.*
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QREFRESH MATERIALIZED VIEW dump_test.matview_second;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xms,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like =>
{ %full_runs, %dump_test_schema_runs, section_post_data => 1, },
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
unlike => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
binary_upgrade => 1,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
exclude_dump_test_schema => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
schema_only => 1,
},
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# FIXME
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'REFRESH MATERIALIZED VIEW matview_third' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QREFRESH MATERIALIZED VIEW dump_test.matview_third;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xms,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# FIXME
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'REFRESH MATERIALIZED VIEW matview_fourth' => {
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QREFRESH MATERIALIZED VIEW dump_test.matview_fourth;\E
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
/xms,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => {},
},
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
Correctly dump database and tablespace ACLs Dump out the appropriate GRANT/REVOKE commands for databases and tablespaces from pg_dumpall to replicate what the current state is. This was broken during the changes to buildACLCommands for 9.6+ servers for pg_init_privs.
2016-07-17 15:04:46 +02:00
'REVOKE CONNECT ON DATABASE dump_test FROM public' => {
create_order => 49,
create_sql => 'REVOKE CONNECT ON DATABASE dump_test FROM public;',
Final pgindent + perltidy run for 9.6.
2016-08-15 19:42:51 +02:00
regexp => qr/^
Correctly dump database and tablespace ACLs Dump out the appropriate GRANT/REVOKE commands for databases and tablespaces from pg_dumpall to replicate what the current state is. This was broken during the changes to buildACLCommands for 9.6+ servers for pg_init_privs.
2016-07-17 15:04:46 +02:00
\QREVOKE CONNECT,TEMPORARY ON DATABASE dump_test FROM PUBLIC;\E\n
Fix pg_dumpall regression test to be locale-independent. The expected results in commit b4fc64578 seem to have been generated in a non-C locale, which just points up the fact that the ORDER BY clause was locale-sensitive. Per buildfarm.
2016-10-13 16:46:22 +02:00
\QGRANT TEMPORARY ON DATABASE dump_test TO PUBLIC;\E\n
\QGRANT CREATE ON DATABASE dump_test TO regress_dump_test_role;\E
Correctly dump database and tablespace ACLs Dump out the appropriate GRANT/REVOKE commands for databases and tablespaces from pg_dumpall to replicate what the current state is. This was broken during the changes to buildACLCommands for 9.6+ servers for pg_init_privs.
2016-07-17 15:04:46 +02:00
/xm,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
like => { pg_dumpall_dbprivs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'REVOKE EXECUTE ON FUNCTION pg_sleep() FROM public' => {
create_order => 15,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'REVOKE EXECUTE ON FUNCTION pg_sleep(float8)
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
FROM public;',
regexp => qr/^
Avoid using unsafe search_path settings during dump and restore. Historically, pg_dump has "set search_path = foo, pg_catalog" when dumping an object in schema "foo", and has also caused that setting to be used while restoring the object. This is problematic because functions and operators in schema "foo" could capture references meant to refer to pg_catalog entries, both in the queries issued by pg_dump and those issued during the subsequent restore run. That could result in dump/restore misbehavior, or in privilege escalation if a nefarious user installs trojan-horse functions or operators. This patch changes pg_dump so that it does not change the search_path dynamically. The emitted restore script sets the search_path to what was used at dump time, and then leaves it alone thereafter. Created objects are placed in the correct schema, regardless of the active search_path, by dint of schema-qualifying their names in the CREATE commands, as well as in subsequent ALTER and ALTER-like commands. Since this change requires a change in the behavior of pg_restore when processing an archive file made according to this new convention, bump the archive file version number; old versions of pg_restore will therefore refuse to process files made with new versions of pg_dump. Security: CVE-2018-1058
2018-02-26 16:18:21 +01:00
\QREVOKE ALL ON FUNCTION pg_catalog.pg_sleep(double precision) FROM PUBLIC;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_privs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'REVOKE SELECT ON TABLE pg_proc FROM public' => {
create_order => 45,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'REVOKE SELECT ON TABLE pg_proc FROM public;',
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
regexp =>
Tighten some regexes with proper character escaping in pg_dump TAP tests Some tests have been using regular expressions which have been lax in escaping dots, which may cause tests to pass when they should not. This make the whole set of tests more robust where needed. Author: David Rowley Reviewed-by: Daniel Gustafsson, Michael Paquier Discussion: https://postgr.es/m/CAKJS1f9jD8aVo1BTH+Vgwd=f-ynbuRVrS90XbWMT6UigaOQJTA@mail.gmail.com
2019-02-06 09:33:55 +01:00
qr/^\QREVOKE SELECT ON TABLE pg_catalog.pg_proc FROM PUBLIC;\E/m,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_privs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'REVOKE CREATE ON SCHEMA public FROM public' => {
create_order => 16,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'REVOKE CREATE ON SCHEMA public FROM public;',
regexp => qr/^
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
\QREVOKE ALL ON SCHEMA public FROM PUBLIC;\E
\n\QGRANT USAGE ON SCHEMA public TO PUBLIC;\E
In new pg_dump TAP tests, remove trailing "$" from regexps using /m. It emerges that some Perl versions before 5.8.9 have a bug with regexps that use the /m flag and contain "$". This is the reason why jacana is still failing on HEAD, and I was able to duplicate the failure on prairiedog's host. There's no real need for "$" in these patterns, since they are already matching through the statement-terminating semicolons (or matching an explicit \n in some cases). So just remove it. Note: the reason jacana hasn't actually reported any failures in the last little while is that the way the pg_dump TAP tests are set up, any failure of this sort results in echoing the entire pg_dump dump output to stderr. Since there were about a hundred such failures, that resulted in a 30MB log file which choked the buildfarm upload script. There is room for improvement here :-(. Per off-list discussion with Andrew and Stephen.
2016-05-07 22:36:50 +02:00
/xm,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
like => { %full_runs, section_pre_data => 1, },
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
unlike => { no_privs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'REVOKE USAGE ON LANGUAGE plpgsql FROM public' => {
create_order => 16,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
create_sql => 'REVOKE USAGE ON LANGUAGE plpgsql FROM public;',
regexp => qr/^REVOKE ALL ON LANGUAGE plpgsql FROM PUBLIC;/m,
like => {
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
%full_runs,
%dump_test_schema_runs,
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
only_dump_test_table => 1,
role => 1,
Restrict vertical tightness to parentheses in Perl code The vertical tightness settings collapse vertical whitespace between opening and closing brackets (parentheses, square brakets and braces). This can make data structures in particular harder to read, and is not very consistent with our style in non-Perl code. This patch restricts that setting to parentheses only, and reformats all the perl code accordingly. Not applying this to parentheses has some unfortunate effects, so the consensus is to keep the setting for parentheses and not for the others. The diff for this patch does highlight some places where structures should have trailing commas. They can be added manually, as there is no automatic tool to do so. Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 16:14:46 +02:00
section_pre_data => 1,
},
unlike => { no_privs => 1, },
},
Clean up pg_dump tests, re-enable BLOB testing Add a loop to check that each test covers all of the pg_dump runs. We (I) had been a bit sloppy when adding new runs and not making sure to mark if they should be under like or unlike for each test, this loop makes sure that the test system will complain if any are forgotten in the future. The loop also correctly handles the 'catch all' cases, which are used to avoid running unnecessary specific checks when a single catch-all can be done (eg: a no-acl run should not have any GRANT commands). Also, re-enable the testing of blobs, but use lo_from_bytea() instead of trying to be cute and writing out to a file and then reading it back in with psql, which proved to be difficult for some buildfarm members. This allows us to add support for testing the --no-blobs option which will be getting added shortly, provided the buildfarm doesn't blow up on this.
2016-11-18 20:21:33 +01:00
tableam: Add pg_dump support. This adds pg_dump support for table AMs in a similar manner to how tablespaces are handled. That is, instead of specifying the AM for every CREATE TABLE etc, emit SET default_table_access_method statements. That makes it easier to change the AM for all/most tables in a dump, and allows restore to succeed even if some AM is not available. This increases the dump archive version, as a tables/matview's AM needs to be tracked therein. Author: Dimitri Dolgov, Andres Freund Discussion: https://postgr.es/m/20180703070645.wchpu5muyto5n647@alap3.anarazel.de https://postgr.es/m/20190304234700.w5tmhducs5wxgzls@alap3.anarazel.de
2019-03-06 18:54:38 +01:00
'CREATE ACCESS METHOD regress_test_table_am' => {
create_order => 11,
create_sql => 'CREATE ACCESS METHOD regress_table_am TYPE TABLE HANDLER heap_tableam_handler;',
regexp => qr/^
\QCREATE ACCESS METHOD regress_table_am TYPE TABLE HANDLER heap_tableam_handler;\E
\n/xm,
like => {
%full_runs,
section_pre_data => 1,
},
},
# It's a bit tricky to ensure that the proper SET of default table
# AM occurs. To achieve that we create a table with the standard
# AM, test AM, standard AM. That guarantees that there needs to be
# a SET interspersed. Then use a regex that prevents interspersed
# SET ...; statements, followed by the exptected CREATE TABLE. Not
# pretty, but seems hard to do better in this framework.
'CREATE TABLE regress_pg_dump_table_am' => {
create_order => 12,
create_sql => '
CREATE TABLE dump_test.regress_pg_dump_table_am_0() USING heap;
CREATE TABLE dump_test.regress_pg_dump_table_am_1 (col1 int) USING regress_table_am;
CREATE TABLE dump_test.regress_pg_dump_table_am_2() USING heap;',
regexp => qr/^
\QSET default_table_access_method = regress_table_am;\E
(\n(?!SET[^;]+;)[^\n]*)*
\n\QCREATE TABLE dump_test.regress_pg_dump_table_am_1 (\E
\n\s+\Qcol1 integer\E
\n\);/xm,
like => {
%full_runs,
%dump_test_schema_runs,
section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1},
},
'CREATE MATERIALIZED VIEW regress_pg_dump_matview_am' => {
create_order => 13,
create_sql => '
CREATE MATERIALIZED VIEW dump_test.regress_pg_dump_matview_am_0 USING heap AS SELECT 1;
CREATE MATERIALIZED VIEW dump_test.regress_pg_dump_matview_am_1
USING regress_table_am AS SELECT count(*) FROM pg_class;
CREATE MATERIALIZED VIEW dump_test.regress_pg_dump_matview_am_2 USING heap AS SELECT 1;',
regexp => qr/^
\QSET default_table_access_method = regress_table_am;\E
(\n(?!SET[^;]+;)[^\n]*)*
\QCREATE MATERIALIZED VIEW dump_test.regress_pg_dump_matview_am_1 AS\E
\n\s+\QSELECT count(*) AS count\E
\n\s+\QFROM pg_class\E
\n\s+\QWITH NO DATA;\E\n/xm,
like => {
%full_runs,
%dump_test_schema_runs,
section_pre_data => 1,
},
unlike => { exclude_dump_test_schema => 1},
}
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
);
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
#########################################
# Create a PG instance to test actually dumping from
my $node = get_new_node('main');
$node->init;
$node->start;
my $port = $node->port;
pg_dump: Skip COLLATION-related regression tests Not every platform supports non-default collations, as pointed out by the buildfarm, so skip collation-related regression tests in pg_dump when they aren't supported.
2017-03-19 21:56:14 +01:00
# We need to see if this system supports CREATE COLLATION or not
# If it doesn't then we will skip all the COLLATION-related tests.
my $collation_support = 0;
my $collation_check_stderr;
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
$node->psql(
'postgres',
"CREATE COLLATION testing FROM \"C\"; DROP COLLATION testing;",
on_error_stop => 0,
stderr => \$collation_check_stderr);
pg_dump: Skip COLLATION-related regression tests Not every platform supports non-default collations, as pointed out by the buildfarm, so skip collation-related regression tests in pg_dump when they aren't supported.
2017-03-19 21:56:14 +01:00
if ($collation_check_stderr !~ /ERROR: /)
{
$collation_support = 1;
}
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
# Create a second database for certain tests to work against
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
$node->psql('postgres', 'create database regress_pg_dump_test;');
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
# Start with number of command_fails_like()*2 tests below (each
# command_fails_like is actually 2 tests)
my $num_tests = 12;
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
foreach my $run (sort keys %pgdump_runs)
{
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
my $test_key = $run;
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
my $run_db = 'postgres';
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
if (defined($pgdump_runs{$run}->{database}))
{
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
$run_db = $pgdump_runs{$run}->{database};
}
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
# Each run of pg_dump is a test itself
$num_tests++;
# If there is a restore cmd, that's another test
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
if ($pgdump_runs{$run}->{restore_cmd})
{
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
$num_tests++;
}
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
if ($pgdump_runs{$run}->{test_key})
{
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
$test_key = $pgdump_runs{$run}->{test_key};
}
# Then count all the tests run against each run
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
foreach my $test (sort keys %tests)
{
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
# postgres is the default database, if it isn't overridden
my $test_db = 'postgres';
# Specific tests can override the database to use
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
if (defined($tests{$test}->{database}))
{
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
$test_db = $tests{$test}->{database};
}
# The database to test against needs to match the database the run is
# for, so skip combinations where they don't match up.
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
if ($run_db ne $test_db)
{
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
next;
}
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
pg_dump: Skip COLLATION-related regression tests Not every platform supports non-default collations, as pointed out by the buildfarm, so skip collation-related regression tests in pg_dump when they aren't supported.
2017-03-19 21:56:14 +01:00
# Skip any collation-related commands if there is no collation support
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
if (!$collation_support && defined($tests{$test}->{collation}))
{
pg_dump: Skip COLLATION-related regression tests Not every platform supports non-default collations, as pointed out by the buildfarm, so skip collation-related regression tests in pg_dump when they aren't supported.
2017-03-19 21:56:14 +01:00
next;
}
perltidy: Add option --nooutdent-long-comments
2018-04-26 20:13:46 +02:00
# If there is a like entry, but no unlike entry, then we will test the like case
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
{
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
$num_tests++;
}
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
else
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
{
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# We will test everything that isn't a 'like'
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
$num_tests++;
}
}
}
plan tests => $num_tests;
#########################################
# Set up schemas, tables, etc, to be dumped.
# Build up the create statements
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
my %create_sql = ();
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
foreach my $test (
sort {
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
if ($tests{$a}->{create_order} and $tests{$b}->{create_order})
{
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
$tests{$a}->{create_order} <=> $tests{$b}->{create_order};
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
}
elsif ($tests{$a}->{create_order})
{
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
-1;
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
}
elsif ($tests{$b}->{create_order})
{
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
1;
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
}
else
{
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
0;
}
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
} keys %tests)
{
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
my $test_db = 'postgres';
Final pgindent + perltidy run for v10.
2017-08-14 23:29:33 +02:00
if (defined($tests{$test}->{database}))
{
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
$test_db = $tests{$test}->{database};
}
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
if ($tests{$test}->{create_sql})
{
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
pg_dump: Skip COLLATION-related regression tests Not every platform supports non-default collations, as pointed out by the buildfarm, so skip collation-related regression tests in pg_dump when they aren't supported.
2017-03-19 21:56:14 +01:00
# Skip any collation-related commands if there is no collation support
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
if (!$collation_support && defined($tests{$test}->{collation}))
{
pg_dump: Skip COLLATION-related regression tests Not every platform supports non-default collations, as pointed out by the buildfarm, so skip collation-related regression tests in pg_dump when they aren't supported.
2017-03-19 21:56:14 +01:00
next;
}
pg_dump/t/002: append terminating semicolon to SQL commands It's easy to overlook the need for one, and its lack is annoying for the next developer wanting to create a new test. Rather than expect every individual command to add the semicolon, just append one automatically. Discussion: http://postgr.es/m/20170503172746.rwftidszir67sgk7@alvherre.pgsql
2017-05-03 20:12:09 +02:00
# Add terminating semicolon
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
$create_sql{$test_db} .= $tests{$test}->{create_sql} . ";";
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
}
}
# Send the combined set of commands to psql
Do not require 'public' to exist for pg_dump -c Commit 330b84d8c4 didn't contemplate the case where the public schema has been dropped and introduced a query which fails when there is no public schema into pg_dump (when used with -c). Adjust the query used by pg_dump to handle the case where the public schema doesn't exist and add tests to check that such a case no longer fails. Back-patch the specific fix to 9.6, as the prior commit was. Adding tests for this case involved adding support to the pg_dump TAP tests to work with multiple databases, which, while not a large change, is a bit much to back-patch, so that's only done in master. Addresses bug #14650 Discussion: https://www.postgresql.org/message-id/20170512181801.1795.47483%40wrigleys.postgresql.org
2017-06-28 16:33:57 +02:00
foreach my $db (sort keys %create_sql)
{
$node->safe_psql($db, $create_sql{$db});
}
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
#########################################
# Test connecting to a non-existent database
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
[ 'pg_dump', '-p', "$port", 'qqq' ],
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
qr/\Qpg_dump: [archiver (db)] connection to database "qqq" failed: FATAL: database "qqq" does not exist\E/,
'pg_dump: [archiver (db)] connection to database "qqq" failed: FATAL: database "qqq" does not exist'
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
);
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
#########################################
# Test connecting with an unprivileged user
command_fails_like(
[ 'pg_dump', '-p', "$port", '--role=regress_dump_test_role' ],
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
qr/\Qpg_dump: [archiver (db)] query failed: ERROR: permission denied for\E/,
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
'pg_dump: [archiver (db)] query failed: ERROR: permission denied for');
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
#########################################
# Test dumping a non-existent schema, table, and patterns with --strict-names
command_fails_like(
[ 'pg_dump', '-p', "$port", '-n', 'nonexistant' ],
qr/\Qpg_dump: no matching schemas were found\E/,
'pg_dump: no matching schemas were found');
command_fails_like(
[ 'pg_dump', '-p', "$port", '-t', 'nonexistant' ],
qr/\Qpg_dump: no matching tables were found\E/,
'pg_dump: no matching tables were found');
command_fails_like(
[ 'pg_dump', '-p', "$port", '--strict-names', '-n', 'nonexistant*' ],
qr/\Qpg_dump: no matching schemas were found for pattern\E/,
'pg_dump: no matching schemas were found for pattern');
command_fails_like(
[ 'pg_dump', '-p', "$port", '--strict-names', '-t', 'nonexistant*' ],
qr/\Qpg_dump: no matching tables were found for pattern\E/,
'pg_dump: no matching tables were found for pattern');
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
#########################################
# Run all runs
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
foreach my $run (sort keys %pgdump_runs)
{
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
my $test_key = $run;
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
my $run_db = 'postgres';
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
$node->command_ok(\@{ $pgdump_runs{$run}->{dump_cmd} },
"$run: pg_dump runs");
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
if ($pgdump_runs{$run}->{restore_cmd})
{
$node->command_ok(\@{ $pgdump_runs{$run}->{restore_cmd} },
"$run: pg_restore runs");
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
}
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
if ($pgdump_runs{$run}->{test_key})
{
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
$test_key = $pgdump_runs{$run}->{test_key};
}
my $output_file = slurp_file("$tempdir/${run}.sql");
#########################################
# Run all tests where this run is included
# as either a 'like' or 'unlike' test.
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
foreach my $test (sort keys %tests)
{
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
my $test_db = 'postgres';
if (defined($pgdump_runs{$run}->{database}))
{
$run_db = $pgdump_runs{$run}->{database};
}
if (defined($tests{$test}->{database}))
{
$test_db = $tests{$test}->{database};
}
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
pg_dump: Skip COLLATION-related regression tests Not every platform supports non-default collations, as pointed out by the buildfarm, so skip collation-related regression tests in pg_dump when they aren't supported.
2017-03-19 21:56:14 +01:00
# Skip any collation-related commands if there is no collation support
Post-PG 10 beta1 pgperltidy run
2017-05-18 01:01:23 +02:00
if (!$collation_support && defined($tests{$test}->{collation}))
{
pg_dump: Skip COLLATION-related regression tests Not every platform supports non-default collations, as pointed out by the buildfarm, so skip collation-related regression tests in pg_dump when they aren't supported.
2017-03-19 21:56:14 +01:00
next;
}
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
if ($run_db ne $test_db)
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
{
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
next;
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
}
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
# Run the test listed as a like, unless it is specifically noted
# as an unlike (generally due to an explicit exclusion or similar).
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
if ($tests{$test}->{like}->{$test_key}
&& !defined($tests{$test}->{unlike}->{$test_key}))
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
{
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
if (!ok($output_file =~ $tests{$test}->{regexp},
"$run: should dump $test"))
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
{
diag("Review $run results in $tempdir");
}
}
else
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
{
Reindent Perl files with perltidy version 20170521. Discussion: https://postgr.es/m/CABUevEzK3cNiHZQ18f5tK0guoT+cN_jWeVzhYYxY=r+1Q3SmoA@mail.gmail.com
2018-04-25 20:00:19 +02:00
if (!ok($output_file !~ $tests{$test}->{regexp},
"$run: should not dump $test"))
Rewrite pg_dump TAP tests This reworks how the tests to run are defined. Instead of having to define all runs for all tests, we define those tests which should pass (generally using one of the defined broad hashes), add in any which should be specific for this test, and exclude any specific runs that shouldn't pass for this test. This ends up removing some 4k+ lines (more than half the file) but, more importantly, greatly simplifies the way runs-to-be-tested are defined. As discussed in the updated comments, for example, take the test which does CREATE TABLE test_table. That CREATE TABLE should show up in all 'full' runs of pg_dump, except those cases where 'test_table' is excluded, of course, and that's exactly how the test gets defined now (modulo a few other related cases, like where we dump only that table, or we dump the schema it's in, or we exclude the schema it's in): like => { %full_runs, %dump_test_schema_runs, only_dump_test_table => 1, section_pre_data => 1, }, unlike => { exclude_dump_test_schema => 1, exclude_test_table => 1, }, }, Next, we no longer expect every run to be listed for every test. If a run is listed in 'like' (directly or through a hash) then it's a 'like', unless it's listed in 'unlike' in which case it's an 'unlike'. If it isn't listed in either, then it's considered an 'unlike' automatically. Lastly, this changes the code to no longer use like/unlike but rather to use 'ok()' with 'diag()' which allows much more control over what gets spit out to the screen. Gone are the days of the entire dump being sent to the console, now you'll just get a couple of lines for each failing test which say the test that failed and the run that it failed on. This covers both the pg_dump TAP tests in src/bin/pg_dump and those in src/test/modules/test_pg_dump.
2018-04-04 21:26:51 +02:00
{
diag("Review $run results in $tempdir");
}
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
}
}
}
#########################################
# Stop the database instance, which will be removed at the end of the tests.
$node->stop('fast');