rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/bin/pg_dump/t/001_basic.pl

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

230 lines
8.5 KiB
Perl
Raw Normal View History

Add a copyright notice to perl files lacking one.
2021-05-07 16:56:14 +02:00
Update copyright for 2023 Backpatch-through: 11
2023-01-02 21:00:37 +01:00
# Copyright (c) 2021-2023, PostgreSQL Global Development Group
Add a copyright notice to perl files lacking one.
2021-05-07 16:56:14 +02:00
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
use strict;
use warnings;
Move Perl test modules to a better namespace The five modules in our TAP test framework all had names in the top level namespace. This is unwise because, even though we're not exporting them to CPAN, the names can leak, for example if they are exported by the RPM build process. We therefore move the modules to the PostgreSQL::Test namespace. In the process PostgresNode is renamed to Cluster, and TestLib is renamed to Utils. PostgresVersion becomes simply PostgreSQL::Version, to avoid possible confusion about what it's the version of. Discussion: https://postgr.es/m/aede93a4-7d92-ef26-398f-5094944c2504@dunslane.net Reviewed by Erik Rijkers and Michael Paquier
2021-10-24 16:28:19 +02:00
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
Replace Test::More plans with done_testing Rather than doing manual book keeping to plan the number of tests to run in each TAP suite, conclude each run with done_testing() summing up the the number of tests that ran. This removes the need for maintaning and updating the plan count at the expense of an accurate count of remaining during the test suite runtime. This patch has been discussed a number of times, often in the context of other patches which updates tests, so a larger number of discussions can be found in the archives. Reviewed-by: Julien Rouhaud <rjuju123@gmail.com> Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/DD399313-3D56-4666-8079-88949DAC870F@yesql.se
2022-02-11 20:54:44 +01:00
use Test::More;
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
Move Perl test modules to a better namespace The five modules in our TAP test framework all had names in the top level namespace. This is unwise because, even though we're not exporting them to CPAN, the names can leak, for example if they are exported by the RPM build process. We therefore move the modules to the PostgreSQL::Test namespace. In the process PostgresNode is renamed to Cluster, and TestLib is renamed to Utils. PostgresVersion becomes simply PostgreSQL::Version, to avoid possible confusion about what it's the version of. Discussion: https://postgr.es/m/aede93a4-7d92-ef26-398f-5094944c2504@dunslane.net Reviewed by Erik Rijkers and Michael Paquier
2021-10-24 16:28:19 +02:00
my $tempdir = PostgreSQL::Test::Utils::tempdir;
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
#########################################
# Basic checks
program_help_ok('pg_dump');
program_version_ok('pg_dump');
program_options_handling_ok('pg_dump');
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
program_help_ok('pg_restore');
program_version_ok('pg_restore');
program_options_handling_ok('pg_restore');
program_help_ok('pg_dumpall');
program_version_ok('pg_dumpall');
program_options_handling_ok('pg_dumpall');
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
#########################################
# Test various invalid options and disallowed combinations
# Doesn't require a PG instance to be set up, so do this first.
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
[ 'pg_dump', 'qqq', 'abc' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dump: error: too many command-line arguments (first is "abc")\E/,
'pg_dump: too many command-line arguments');
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
[ 'pg_restore', 'qqq', 'abc' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_restore: error: too many command-line arguments (first is "abc")\E/,
'pg_restore: too many command-line arguments');
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
[ 'pg_dumpall', 'qqq', 'abc' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dumpall: error: too many command-line arguments (first is "qqq")\E/,
'pg_dumpall: too many command-line arguments');
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
[ 'pg_dump', '-s', '-a' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dump: error: options -s\/--schema-only and -a\/--data-only cannot be used together\E/,
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'pg_dump: options -s/--schema-only and -a/--data-only cannot be used together'
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
);
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
pg_dump: Allow dumping data of specific foreign servers The new command-line switch --include-foreign-data=PATTERN lets the user specify foreign servers from which to dump foreign table data. This can be refined by further inclusion/exclusion switches, so that the user has full control over which tables to dump. A limitation is that this doesn't work in combination with parallel dumps, for implementation reasons. This might be lifted in the future, but requires shuffling some code around. Author: Luis Carril <luis.carril@swarm64.com> Reviewed-by: Daniel Gustafsson <daniel@yesql.se> Reviewed-by: Surafel Temesgen <surafel3000@gmail.com> Reviewed-by: vignesh C <vignesh21@gmail.com> Reviewed-by: Álvaro Herrera <alvherre@2ndQuadrant.com> Discussion: https://postgr.es/m/LEJPR01MB0185483C0079D2F651B16231E7FC0@LEJPR01MB0185.DEUPRD01.PROD.OUTLOOK.DE
2020-03-25 17:19:31 +01:00
command_fails_like(
[ 'pg_dump', '-s', '--include-foreign-data=xxx' ],
qr/\Qpg_dump: error: options -s\/--schema-only and --include-foreign-data cannot be used together\E/,
'pg_dump: options -s/--schema-only and --include-foreign-data cannot be used together'
);
command_fails_like(
[ 'pg_dump', '-j2', '--include-foreign-data=xxx' ],
qr/\Qpg_dump: error: option --include-foreign-data is not supported with parallel backup\E/,
'pg_dump: option --include-foreign-data is not supported with parallel backup'
);
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
pg_restore: Require "-f -" to mean stdout The previous convention that stdout was selected by default when nothing is specified was just too error-prone. After a suggestion from Andrew Gierth. Author: Euler Taveira Reviewed-by: Yoshikazu Imai, José Arthur Benetasso Villanova Discussion: https://postgr.es/m/87sgwrmhdv.fsf@news-spur.riddles.org.uk
2019-04-04 21:34:58 +02:00
['pg_restore'],
qr{\Qpg_restore: error: one of -d/--dbname and -f/--file must be specified\E},
'pg_restore: error: one of -d/--dbname and -f/--file must be specified');
command_fails_like(
[ 'pg_restore', '-s', '-a', '-f -' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_restore: error: options -s\/--schema-only and -a\/--data-only cannot be used together\E/,
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'pg_restore: options -s/--schema-only and -a/--data-only cannot be used together'
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
);
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
[ 'pg_restore', '-d', 'xxx', '-f', 'xxx' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_restore: error: options -d\/--dbname and -f\/--file cannot be used together\E/,
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_restore: options -d/--dbname and -f/--file cannot be used together');
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
[ 'pg_dump', '-c', '-a' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dump: error: options -c\/--clean and -a\/--data-only cannot be used together\E/,
Add TAP tests for pg_dump This TAP test suite will create a new cluster, populate it based on the 'create_sql' values in the '%tests' hash, run all of the runs defined in the '%pgdump_runs' hash, and then for each test in the '%tests' hash, compare each run's output the the regular expression defined for the test under the 'like' and 'unlike' functions, as appropriate. While this test suite covers a fair bit of ground (67% of pg_dump.c and quite a bit of the other files in src/bin/pg_dump), there is still quite a bit which remains to be added to provide better code coverage. Still, this is quite a bit better than we had, and has found a few bugs already (note that the CREATE TRANSFORM test is commented out, as it is currently failing). Idea for using the TAP system from Tom, though all of the code is mine.
2016-05-06 20:06:50 +02:00
'pg_dump: options -c/--clean and -a/--data-only cannot be used together');
Finish pgindent run for 9.6: Perl files.
2016-06-12 10:19:56 +02:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
pg_restore: Require "-f -" to mean stdout The previous convention that stdout was selected by default when nothing is specified was just too error-prone. After a suggestion from Andrew Gierth. Author: Euler Taveira Reviewed-by: Yoshikazu Imai, José Arthur Benetasso Villanova Discussion: https://postgr.es/m/87sgwrmhdv.fsf@news-spur.riddles.org.uk
2019-04-04 21:34:58 +02:00
[ 'pg_restore', '-c', '-a', '-f -' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_restore: error: options -c\/--clean and -a\/--data-only cannot be used together\E/,
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'pg_restore: options -c/--clean and -a/--data-only cannot be used together'
);
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
[ 'pg_dump', '--if-exists' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dump: error: option --if-exists requires option -c\/--clean\E/,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'pg_dump: option --if-exists requires option -c/--clean');
command_fails_like(
[ 'pg_dump', '-j3' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dump: error: parallel backup only supported by the directory format\E/,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'pg_dump: parallel backup only supported by the directory format');
Skip trailing whitespaces when parsing integer options strtoint(), via strtol(), would skip leading whitespaces but the same rule was not applied for trailing whitespaces, leading to an inconsistent behavior. Some tests are changed to cover more this area. Author: Michael Paquier Reviewed-by: Kyotaro Horiguchi Discussion: https://postgr.es/m/YP5Pv0d13Ct+03ve@paquier.xyz
2021-07-27 03:39:05 +02:00
# Note the trailing whitespace for the value of --jobs, that is valid.
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
Skip trailing whitespaces when parsing integer options strtoint(), via strtol(), would skip leading whitespaces but the same rule was not applied for trailing whitespaces, leading to an inconsistent behavior. Some tests are changed to cover more this area. Author: Michael Paquier Reviewed-by: Kyotaro Horiguchi Discussion: https://postgr.es/m/YP5Pv0d13Ct+03ve@paquier.xyz
2021-07-27 03:39:05 +02:00
[ 'pg_dump', '-j', '-1 ' ],
Unify parsing logic for command-line integer options Most of the integer options for command-line binaries now make use of a single routine able to do the job, fixing issues with the detection of sloppy values caused for example by the use of atoi(), that fails on strings beginning with numerical characters with junk trailing characters. This commit cuts down the number of strings requiring translation by 26 per my count, switching the code to have two error types for invalid and out-of-range values instead. Much more could be done here, with float or even int64 options, but int32 was the most appealing case as it is possible to rely on strtol() to do the job reliably. Note that there are some exceptions for now, like pg_ctl or pg_upgrade that use their own logging logic. A couple of negative TAP tests required some adjustments for the new errors generated. pg_dump and pg_restore tracked the maximum number of parallel jobs within the option parsing. The code is refactored a bit to track that in the code dedicated to parallelism instead. Author: Kyotaro Horiguchi, Michael Paquier Reviewed-by: David Rowley, Álvaro Herrera Discussion: https://postgr.es/m/CALj2ACXqdG9WhqVoJ9zYf-iZt7sgK7Szv5USs=he6NnWQ2ofTA@mail.gmail.com
2021-07-24 11:35:03 +02:00
qr/\Qpg_dump: error: -j\/--jobs must be in range\E/,
'pg_dump: -j/--jobs must be in range');
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
[ 'pg_dump', '-F', 'garbage' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dump: error: invalid output format\E/,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'pg_dump: invalid output format');
command_fails_like(
pg_restore: Require "-f -" to mean stdout The previous convention that stdout was selected by default when nothing is specified was just too error-prone. After a suggestion from Andrew Gierth. Author: Euler Taveira Reviewed-by: Yoshikazu Imai, José Arthur Benetasso Villanova Discussion: https://postgr.es/m/87sgwrmhdv.fsf@news-spur.riddles.org.uk
2019-04-04 21:34:58 +02:00
[ 'pg_restore', '-j', '-1', '-f -' ],
Unify parsing logic for command-line integer options Most of the integer options for command-line binaries now make use of a single routine able to do the job, fixing issues with the detection of sloppy values caused for example by the use of atoi(), that fails on strings beginning with numerical characters with junk trailing characters. This commit cuts down the number of strings requiring translation by 26 per my count, switching the code to have two error types for invalid and out-of-range values instead. Much more could be done here, with float or even int64 options, but int32 was the most appealing case as it is possible to rely on strtol() to do the job reliably. Note that there are some exceptions for now, like pg_ctl or pg_upgrade that use their own logging logic. A couple of negative TAP tests required some adjustments for the new errors generated. pg_dump and pg_restore tracked the maximum number of parallel jobs within the option parsing. The code is refactored a bit to track that in the code dedicated to parallelism instead. Author: Kyotaro Horiguchi, Michael Paquier Reviewed-by: David Rowley, Álvaro Herrera Discussion: https://postgr.es/m/CALj2ACXqdG9WhqVoJ9zYf-iZt7sgK7Szv5USs=he6NnWQ2ofTA@mail.gmail.com
2021-07-24 11:35:03 +02:00
qr/\Qpg_restore: error: -j\/--jobs must be in range\E/,
'pg_restore: -j/--jobs must be in range');
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
pg_restore: Require "-f -" to mean stdout The previous convention that stdout was selected by default when nothing is specified was just too error-prone. After a suggestion from Andrew Gierth. Author: Euler Taveira Reviewed-by: Yoshikazu Imai, José Arthur Benetasso Villanova Discussion: https://postgr.es/m/87sgwrmhdv.fsf@news-spur.riddles.org.uk
2019-04-04 21:34:58 +02:00
[ 'pg_restore', '--single-transaction', '-j3', '-f -' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_restore: error: cannot specify both --single-transaction and multiple jobs\E/,
perltidy pg_dump TAP tests The pg_dump TAP tests have gotten pretty far from what perltidy thinks they should be, so fix that, and in passing use long-form argument names with arguments passed via "=" in a similar vein to 58da833. No functional changes here, just whitespace and changing runs from "-f" to "--file=", and similar.
2017-01-31 18:42:16 +01:00
'pg_restore: cannot specify both --single-transaction and multiple jobs');
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
Switch pg_dump to use compression specifications Compression specifications are currently used by pg_basebackup and pg_receivewal, and are able to let the user control in an extended way the method and level of compression used. As an effect of this commit, pg_dump's -Z/--compress is now able to use more than just an integer, as of the grammar "method[:detail]". The method can be either "none" or "gzip", and can optionally take a detail string. If the detail string is only an integer, it defines the compression level. A comma-separated list of keywords can also be used method allows for more options, the only keyword supported now is "level". The change is backward-compatible, hence specifying only an integer leads to no compression for a level of 0 and gzip compression when the level is greater than 0. Most of the code changes are straight-forward, as pg_dump was relying on an integer tracking the compression level to check for gzip or no compression. These are changed to use a compression specification and the algorithm stored in it. As of this change, note that the dump format is not bumped because there is no need yet to track the compression algorithm in the TOC entries. Hence, we still rely on the compression level to make the difference when reading them. This will be mandatory once a new compression method is added, though. In order to keep the code simpler when parsing the compression specification, the code is changed so as pg_dump now fails hard when using gzip on -Z/--compress without its support compiled, rather than enforcing no compression without the user knowing about it except through a warning. Like before this commit, archive and custom formats are compressed by default when the code is compiled with gzip, and left uncompressed without gzip. Author: Georgios Kokolatos Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/O4mutIrCES8ZhlXJiMvzsivT7ztAMja2lkdL1LJx6O5f22I2W8PBIeLKz7mDLwxHoibcnRAYJXm1pH4tyUNC4a8eDzLn22a6Pb1S74Niexg=@pm.me
2022-12-02 02:45:02 +01:00
[ 'pg_dump', '--compress', 'garbage' ],
qr/\Qpg_dump: error: unrecognized compression algorithm/,
'pg_dump: invalid --compress');
command_fails_like(
[ 'pg_dump', '--compress', 'none:1' ],
qr/\Qpg_dump: error: invalid compression specification: compression algorithm "none" does not accept a compression level\E/,
'pg_dump: invalid compression specification: compression algorithm "none" does not accept a compression level'
);
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
Add TAP test in pg_dump with --format=tar and --compress This combination of options has never been supported, and it has never been checked in the regression tests. When building the code without zlib support, pg_dump is allowed to run and it generates a warning to inform that any contents are dumped as uncompressed. The tests added by this commit check both behaviors. Author: Georgios Kokolatos, Rachel Heaton Discussion: https://postgr.es/m/faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss=@protonmail.com
2022-03-30 06:34:01 +02:00
if (check_pg_config("#define HAVE_LIBZ 1"))
{
Switch pg_dump to use compression specifications Compression specifications are currently used by pg_basebackup and pg_receivewal, and are able to let the user control in an extended way the method and level of compression used. As an effect of this commit, pg_dump's -Z/--compress is now able to use more than just an integer, as of the grammar "method[:detail]". The method can be either "none" or "gzip", and can optionally take a detail string. If the detail string is only an integer, it defines the compression level. A comma-separated list of keywords can also be used method allows for more options, the only keyword supported now is "level". The change is backward-compatible, hence specifying only an integer leads to no compression for a level of 0 and gzip compression when the level is greater than 0. Most of the code changes are straight-forward, as pg_dump was relying on an integer tracking the compression level to check for gzip or no compression. These are changed to use a compression specification and the algorithm stored in it. As of this change, note that the dump format is not bumped because there is no need yet to track the compression algorithm in the TOC entries. Hence, we still rely on the compression level to make the difference when reading them. This will be mandatory once a new compression method is added, though. In order to keep the code simpler when parsing the compression specification, the code is changed so as pg_dump now fails hard when using gzip on -Z/--compress without its support compiled, rather than enforcing no compression without the user knowing about it except through a warning. Like before this commit, archive and custom formats are compressed by default when the code is compiled with gzip, and left uncompressed without gzip. Author: Georgios Kokolatos Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/O4mutIrCES8ZhlXJiMvzsivT7ztAMja2lkdL1LJx6O5f22I2W8PBIeLKz7mDLwxHoibcnRAYJXm1pH4tyUNC4a8eDzLn22a6Pb1S74Niexg=@pm.me
2022-12-02 02:45:02 +01:00
command_fails_like(
[ 'pg_dump', '-Z', '15' ],
qr/\Qpg_dump: error: invalid compression specification: compression algorithm "gzip" expects a compression level between 1 and 9 (default at -1)\E/,
'pg_dump: invalid compression specification: must be in range');
Add TAP test in pg_dump with --format=tar and --compress This combination of options has never been supported, and it has never been checked in the regression tests. When building the code without zlib support, pg_dump is allowed to run and it generates a warning to inform that any contents are dumped as uncompressed. The tests added by this commit check both behaviors. Author: Georgios Kokolatos, Rachel Heaton Discussion: https://postgr.es/m/faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss=@protonmail.com
2022-03-30 06:34:01 +02:00
command_fails_like(
[ 'pg_dump', '--compress', '1', '--format', 'tar' ],
qr/\Qpg_dump: error: compression is not supported by tar archive format\E/,
'pg_dump: compression is not supported by tar archive format');
Switch pg_dump to use compression specifications Compression specifications are currently used by pg_basebackup and pg_receivewal, and are able to let the user control in an extended way the method and level of compression used. As an effect of this commit, pg_dump's -Z/--compress is now able to use more than just an integer, as of the grammar "method[:detail]". The method can be either "none" or "gzip", and can optionally take a detail string. If the detail string is only an integer, it defines the compression level. A comma-separated list of keywords can also be used method allows for more options, the only keyword supported now is "level". The change is backward-compatible, hence specifying only an integer leads to no compression for a level of 0 and gzip compression when the level is greater than 0. Most of the code changes are straight-forward, as pg_dump was relying on an integer tracking the compression level to check for gzip or no compression. These are changed to use a compression specification and the algorithm stored in it. As of this change, note that the dump format is not bumped because there is no need yet to track the compression algorithm in the TOC entries. Hence, we still rely on the compression level to make the difference when reading them. This will be mandatory once a new compression method is added, though. In order to keep the code simpler when parsing the compression specification, the code is changed so as pg_dump now fails hard when using gzip on -Z/--compress without its support compiled, rather than enforcing no compression without the user knowing about it except through a warning. Like before this commit, archive and custom formats are compressed by default when the code is compiled with gzip, and left uncompressed without gzip. Author: Georgios Kokolatos Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/O4mutIrCES8ZhlXJiMvzsivT7ztAMja2lkdL1LJx6O5f22I2W8PBIeLKz7mDLwxHoibcnRAYJXm1pH4tyUNC4a8eDzLn22a6Pb1S74Niexg=@pm.me
2022-12-02 02:45:02 +01:00
command_fails_like(
[ 'pg_dump', '-Z', 'gzip:nonInt' ],
qr/\Qpg_dump: error: invalid compression specification: unrecognized compression option: "nonInt"\E/,
'pg_dump: invalid compression specification: must be an integer');
Add TAP test in pg_dump with --format=tar and --compress This combination of options has never been supported, and it has never been checked in the regression tests. When building the code without zlib support, pg_dump is allowed to run and it generates a warning to inform that any contents are dumped as uncompressed. The tests added by this commit check both behaviors. Author: Georgios Kokolatos, Rachel Heaton Discussion: https://postgr.es/m/faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss=@protonmail.com
2022-03-30 06:34:01 +02:00
}
else
{
# --jobs > 1 forces an error with tar format.
command_fails_like(
Switch pg_dump to use compression specifications Compression specifications are currently used by pg_basebackup and pg_receivewal, and are able to let the user control in an extended way the method and level of compression used. As an effect of this commit, pg_dump's -Z/--compress is now able to use more than just an integer, as of the grammar "method[:detail]". The method can be either "none" or "gzip", and can optionally take a detail string. If the detail string is only an integer, it defines the compression level. A comma-separated list of keywords can also be used method allows for more options, the only keyword supported now is "level". The change is backward-compatible, hence specifying only an integer leads to no compression for a level of 0 and gzip compression when the level is greater than 0. Most of the code changes are straight-forward, as pg_dump was relying on an integer tracking the compression level to check for gzip or no compression. These are changed to use a compression specification and the algorithm stored in it. As of this change, note that the dump format is not bumped because there is no need yet to track the compression algorithm in the TOC entries. Hence, we still rely on the compression level to make the difference when reading them. This will be mandatory once a new compression method is added, though. In order to keep the code simpler when parsing the compression specification, the code is changed so as pg_dump now fails hard when using gzip on -Z/--compress without its support compiled, rather than enforcing no compression without the user knowing about it except through a warning. Like before this commit, archive and custom formats are compressed by default when the code is compiled with gzip, and left uncompressed without gzip. Author: Georgios Kokolatos Reviewed-by: Michael Paquier Discussion: https://postgr.es/m/O4mutIrCES8ZhlXJiMvzsivT7ztAMja2lkdL1LJx6O5f22I2W8PBIeLKz7mDLwxHoibcnRAYJXm1pH4tyUNC4a8eDzLn22a6Pb1S74Niexg=@pm.me
2022-12-02 02:45:02 +01:00
[ 'pg_dump', '--format', 'tar', '-j3' ],
qr/\Qpg_dump: error: parallel backup only supported by the directory format\E/,
'pg_dump: warning: parallel backup not supported by tar format');
command_fails_like(
[ 'pg_dump', '-Z', 'gzip:nonInt', '--format', 'tar', '-j2' ],
qr/\Qpg_dump: error: invalid compression specification: unrecognized compression option\E/,
'pg_dump: invalid compression specification: must be an integer');
Add TAP test in pg_dump with --format=tar and --compress This combination of options has never been supported, and it has never been checked in the regression tests. When building the code without zlib support, pg_dump is allowed to run and it generates a warning to inform that any contents are dumped as uncompressed. The tests added by this commit check both behaviors. Author: Georgios Kokolatos, Rachel Heaton Discussion: https://postgr.es/m/faUNEOpts9vunEaLnmxmG-DldLSg_ql137OC3JYDmgrOMHm1RvvWY2IdBkv_CRxm5spCCb_OmKNk2T03TMm0fBEWveFF9wA1WizPuAgB7Ss=@protonmail.com
2022-03-30 06:34:01 +02:00
}
Add more TAP tests for pg_dump options with range checks This adds two tests for --extra-float-digits and --rows-per-insert, similar to what exists for --compress. Author: Dong Wook Lee Discussion: https://postgr.es/m/CAAcByaJsgrB-qc-ALb0mALprRGLAdmcBap7SZxO4kCAU-JEHcQ@mail.gmail.com
2020-06-13 02:34:38 +02:00
command_fails_like(
[ 'pg_dump', '--extra-float-digits', '-16' ],
Unify parsing logic for command-line integer options Most of the integer options for command-line binaries now make use of a single routine able to do the job, fixing issues with the detection of sloppy values caused for example by the use of atoi(), that fails on strings beginning with numerical characters with junk trailing characters. This commit cuts down the number of strings requiring translation by 26 per my count, switching the code to have two error types for invalid and out-of-range values instead. Much more could be done here, with float or even int64 options, but int32 was the most appealing case as it is possible to rely on strtol() to do the job reliably. Note that there are some exceptions for now, like pg_ctl or pg_upgrade that use their own logging logic. A couple of negative TAP tests required some adjustments for the new errors generated. pg_dump and pg_restore tracked the maximum number of parallel jobs within the option parsing. The code is refactored a bit to track that in the code dedicated to parallelism instead. Author: Kyotaro Horiguchi, Michael Paquier Reviewed-by: David Rowley, Álvaro Herrera Discussion: https://postgr.es/m/CALj2ACXqdG9WhqVoJ9zYf-iZt7sgK7Szv5USs=he6NnWQ2ofTA@mail.gmail.com
2021-07-24 11:35:03 +02:00
qr/\Qpg_dump: error: --extra-float-digits must be in range\E/,
'pg_dump: --extra-float-digits must be in range');
Add more TAP tests for pg_dump options with range checks This adds two tests for --extra-float-digits and --rows-per-insert, similar to what exists for --compress. Author: Dong Wook Lee Discussion: https://postgr.es/m/CAAcByaJsgrB-qc-ALb0mALprRGLAdmcBap7SZxO4kCAU-JEHcQ@mail.gmail.com
2020-06-13 02:34:38 +02:00
command_fails_like(
[ 'pg_dump', '--rows-per-insert', '0' ],
Unify parsing logic for command-line integer options Most of the integer options for command-line binaries now make use of a single routine able to do the job, fixing issues with the detection of sloppy values caused for example by the use of atoi(), that fails on strings beginning with numerical characters with junk trailing characters. This commit cuts down the number of strings requiring translation by 26 per my count, switching the code to have two error types for invalid and out-of-range values instead. Much more could be done here, with float or even int64 options, but int32 was the most appealing case as it is possible to rely on strtol() to do the job reliably. Note that there are some exceptions for now, like pg_ctl or pg_upgrade that use their own logging logic. A couple of negative TAP tests required some adjustments for the new errors generated. pg_dump and pg_restore tracked the maximum number of parallel jobs within the option parsing. The code is refactored a bit to track that in the code dedicated to parallelism instead. Author: Kyotaro Horiguchi, Michael Paquier Reviewed-by: David Rowley, Álvaro Herrera Discussion: https://postgr.es/m/CALj2ACXqdG9WhqVoJ9zYf-iZt7sgK7Szv5USs=he6NnWQ2ofTA@mail.gmail.com
2021-07-24 11:35:03 +02:00
qr/\Qpg_dump: error: --rows-per-insert must be in range\E/,
'pg_dump: --rows-per-insert must be in range');
Add more TAP tests for pg_dump options with range checks This adds two tests for --extra-float-digits and --rows-per-insert, similar to what exists for --compress. Author: Dong Wook Lee Discussion: https://postgr.es/m/CAAcByaJsgrB-qc-ALb0mALprRGLAdmcBap7SZxO4kCAU-JEHcQ@mail.gmail.com
2020-06-13 02:34:38 +02:00
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
pg_restore: Require "-f -" to mean stdout The previous convention that stdout was selected by default when nothing is specified was just too error-prone. After a suggestion from Andrew Gierth. Author: Euler Taveira Reviewed-by: Yoshikazu Imai, José Arthur Benetasso Villanova Discussion: https://postgr.es/m/87sgwrmhdv.fsf@news-spur.riddles.org.uk
2019-04-04 21:34:58 +02:00
[ 'pg_restore', '--if-exists', '-f -' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_restore: error: option --if-exists requires option -c\/--clean\E/,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'pg_restore: option --if-exists requires option -c/--clean');
command_fails_like(
pg_restore: Require "-f -" to mean stdout The previous convention that stdout was selected by default when nothing is specified was just too error-prone. After a suggestion from Andrew Gierth. Author: Euler Taveira Reviewed-by: Yoshikazu Imai, José Arthur Benetasso Villanova Discussion: https://postgr.es/m/87sgwrmhdv.fsf@news-spur.riddles.org.uk
2019-04-04 21:34:58 +02:00
[ 'pg_restore', '-f -', '-F', 'garbage' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_restore: error: unrecognized archive format "garbage";\E/,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'pg_dump: unrecognized archive format');
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
Add pg_dump --on-conflict-do-nothing option. When dumping INSERT statements, optionally add ON CONFLICT DO NOTHING. Author: Surafel Temesgen Reviewed-by: Takeshi Ideriha, Nico Williams, Dilip Kumar Discussion: https://postgr.es/m/CALAY4q-PQ9cOEzs2%2BQHK5ObfF_4QbmBaYXbZx6BGGN66Q-n8FA%40mail.gmail.com
2018-07-13 03:57:03 +02:00
command_fails_like(
[ 'pg_dump', '--on-conflict-do-nothing' ],
Add missing serial commas
2019-07-01 13:07:14 +02:00
qr/pg_dump: error: option --on-conflict-do-nothing requires option --inserts, --rows-per-insert, or --column-inserts/,
pg_dump: allow multiple rows per insert This is useful to speed up loading data in a different database engine. Authors: Surafel Temesgen and David Rowley. Lightly edited by Álvaro. Reviewed-by: Fabien Coelho Discussion: https://postgr.es/m/CALAY4q9kumSdnRBzvRJvSRf2+BH20YmSvzqOkvwpEmodD-xv6g@mail.gmail.com
2019-03-07 13:26:14 +01:00
'pg_dump: --on-conflict-do-nothing requires --inserts, --rows-per-insert, --column-inserts'
);
Add pg_dump --on-conflict-do-nothing option. When dumping INSERT statements, optionally add ON CONFLICT DO NOTHING. Author: Surafel Temesgen Reviewed-by: Takeshi Ideriha, Nico Williams, Dilip Kumar Discussion: https://postgr.es/m/CALAY4q-PQ9cOEzs2%2BQHK5ObfF_4QbmBaYXbZx6BGGN66Q-n8FA%40mail.gmail.com
2018-07-13 03:57:03 +02:00
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
# pg_dumpall command-line argument checks
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
[ 'pg_dumpall', '-g', '-r' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dumpall: error: options -g\/--globals-only and -r\/--roles-only cannot be used together\E/,
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'pg_dumpall: options -g/--globals-only and -r/--roles-only cannot be used together'
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
);
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
[ 'pg_dumpall', '-g', '-t' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dumpall: error: options -g\/--globals-only and -t\/--tablespaces-only cannot be used together\E/,
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'pg_dumpall: options -g/--globals-only and -t/--tablespaces-only cannot be used together'
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
);
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
[ 'pg_dumpall', '-r', '-t' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dumpall: error: options -r\/--roles-only and -t\/--tablespaces-only cannot be used together\E/,
perltidy: Add option --nooutdent-long-quotes
2018-04-26 17:52:52 +02:00
'pg_dumpall: options -r/--roles-only and -t/--tablespaces-only cannot be used together'
Add basic pg_dumpall/pg_restore TAP tests For reasons unknown, pg_dumpall and pg_restore managed to escape the basic set of TAP tests that were added for pg_dump in 6bd356c3, so let's get them added now. A few minor adjustments are also made to the dump/restore tests to improve code coverage for pg_restore/pg_dumpall.
2017-01-06 22:29:31 +01:00
);
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
command_fails_like(
[ 'pg_dumpall', '--if-exists' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dumpall: error: option --if-exists requires option -c\/--clean\E/,
Improve pg_dump regression tests and code coverage These improvements bring the lines-of-code coverage of pg_dump.c up to 87.7% (at least using LCOV 1.12, 1.11 seems to differ slightly). Nearly every function is covered, three of the four which aren't are only called when talking to older PG instances. There is more which can, and should, be done here to improve the coverage but it's past time to see what the buildfarm thinks of this. What has been added: - Coverage for many more command-line options - Use command_fails_like instead of command_exit_is - Operator classes, operator families - Text search configuration, templates, parsers, dictionaries - FDWs, servers, foreign tables - Materialized views - Improved Publications / Subscriptions test (though this needs work, see PG10 open items and tests marked with XXX in 002_pg_dump.pl) - Unlogged tables - Partitioned tables - Additional ACL testing for various object types There is room for improvement, specifically: - Various type-based option (alignment, storage, etc) - Composite type collation - Extra Procedural language functions (inline, validator) - Different function options (SRF, Transform, config, security definer, cost, leakproof) - OpClass options (default, storage, order by, recheck) - OpFamily options (order by, recheck) - Aggregate functions (combinefunc, serialfunc, deserialfunc, etc) - Text Search parser 'headline' - Text Search template 'init' - FDW options (handler, validator, options) - Server options (type, version, options) - User mapping options - Default ACLs for sequences, types - Security labels - View circular dependencies (last function that needs coverage) - Toast table autovacuum options - Replica identity options - Independent indexes (plus marking them as clustered on) - Deferrable / initially deferred constraints - Independent domain constraints There's bits of extension pg_dump'ing also not covered, but those will need to go into test_pg_dump (such as having a filter for config tables). Last, but not least, this approximately halves the number of tests run with 'ok()' by removing the ok()-based checking of if all runs are covered by each test. Instead, 002_pg_dump.pl will just exit out in such a case (with a message in the log file). In general, when adding tests, cover all runs unless there is a very good reason not to (such as adding a 'catch-all' case). With these changes, the resulting output and number of "tests" run is actually reduced.
2017-03-18 18:18:24 +01:00
'pg_dumpall: option --if-exists requires option -c/--clean');
Consolidate cross-option checks in pg_restore This moves one check for conflicting options from the archive restore code to the main function where other similar checks are performed. Also reword the error message to be consistent with other messages. The only option combination impacted is --create specified with --single-transaction, and informing the caller at an early step saves from opening the archive worked on. A TAP test is added for this combination. Author: Daniel Gustafsson Reviewed-by: Fabien Coelho Discussion: https://postgr.es/m/616808BD-4B59-4E6C-97A9-7317F62D5570@yesql.se
2018-10-30 03:38:35 +01:00
command_fails_like(
pg_restore: Require "-f -" to mean stdout The previous convention that stdout was selected by default when nothing is specified was just too error-prone. After a suggestion from Andrew Gierth. Author: Euler Taveira Reviewed-by: Yoshikazu Imai, José Arthur Benetasso Villanova Discussion: https://postgr.es/m/87sgwrmhdv.fsf@news-spur.riddles.org.uk
2019-04-04 21:34:58 +02:00
[ 'pg_restore', '-C', '-1', '-f -' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_restore: error: options -C\/--create and -1\/--single-transaction cannot be used together\E/,
Consolidate cross-option checks in pg_restore This moves one check for conflicting options from the archive restore code to the main function where other similar checks are performed. Also reword the error message to be consistent with other messages. The only option combination impacted is --create specified with --single-transaction, and informing the caller at an early step saves from opening the archive worked on. A TAP test is added for this combination. Author: Daniel Gustafsson Reviewed-by: Fabien Coelho Discussion: https://postgr.es/m/616808BD-4B59-4E6C-97A9-7317F62D5570@yesql.se
2018-10-30 03:38:35 +01:00
'pg_restore: options -C\/--create and -1\/--single-transaction cannot be used together'
);
Add --exclude-database option to pg_dumpall This option functions similarly to pg_dump's --exclude-table option, but for database names. The option can be given once, and the argument can be a pattern including wildcard characters. Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier Discussion: https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
2019-03-01 16:47:44 +01:00
# also fails for -r and -t, but it seems pointless to add more tests for those.
command_fails_like(
[ 'pg_dumpall', '--exclude-database=foo', '--globals-only' ],
Unified logging system for command-line programs This unifies the various ad hoc logging (message printing, error printing) systems used throughout the command-line programs. Features: - Program name is automatically prefixed. - Message string does not end with newline. This removes a common source of inconsistencies and omissions. - Additionally, a final newline is automatically stripped, simplifying use of PQerrorMessage() etc., another common source of mistakes. - I converted error message strings to use %m where possible. - As a result of the above several points, more translatable message strings can be shared between different components and between frontends and backend, without gratuitous punctuation or whitespace differences. - There is support for setting a "log level". This is not meant to be user-facing, but can be used internally to implement debug or verbose modes. - Lazy argument evaluation, so no significant overhead if logging at some level is disabled. - Some color in the messages, similar to gcc and clang. Set PG_COLOR=auto to try it out. Some colors are predefined, but can be customized by setting PG_COLORS. - Common files (common/, fe_utils/, etc.) can handle logging much more simply by just using one API without worrying too much about the context of the calling program, requiring callbacks, or having to pass "progname" around everywhere. - Some programs called setvbuf() to make sure that stderr is unbuffered, even on Windows. But not all programs did that. This is now done centrally. Soft goals: - Reduces vertical space use and visual complexity of error reporting in the source code. - Encourages more deliberate classification of messages. For example, in some cases it wasn't clear without analyzing the surrounding code whether a message was meant as an error or just an info. - Concepts and terms are vaguely aligned with popular logging frameworks such as log4j and Python logging. This is all just about printing stuff out. Nothing affects program flow (e.g., fatal exits). The uses are just too varied to do that. Some existing code had wrappers that do some kind of print-and-exit, and I adapted those. I tried to keep the output mostly the same, but there is a lot of historical baggage to unwind and special cases to consider, and I might not always have succeeded. One significant change is that pg_rewind used to write all error messages to stdout. That is now changed to stderr. Reviewed-by: Donald Dong <xdong@csumb.edu> Reviewed-by: Arthur Zakirov <a.zakirov@postgrespro.ru> Discussion: https://www.postgresql.org/message-id/flat/6a609b43-4f57-7348-6480-bd022f924310@2ndquadrant.com
2019-04-01 14:24:37 +02:00
qr/\Qpg_dumpall: error: option --exclude-database cannot be used together with -g\/--globals-only\E/,
Add --exclude-database option to pg_dumpall This option functions similarly to pg_dump's --exclude-table option, but for database names. The option can be given once, and the argument can be a pattern including wildcard characters. Author: Andrew Dunstan. Reviewd-by: Fabien Coelho and Michael Paquier Discussion: https://postgr.es/m/43a54a47-4aa7-c70e-9ca6-648f436dd6e6@2ndQuadrant.com
2019-03-01 16:47:44 +01:00
'pg_dumpall: option --exclude-database cannot be used together with -g/--globals-only'
);
Replace Test::More plans with done_testing Rather than doing manual book keeping to plan the number of tests to run in each TAP suite, conclude each run with done_testing() summing up the the number of tests that ran. This removes the need for maintaning and updating the plan count at the expense of an accurate count of remaining during the test suite runtime. This patch has been discussed a number of times, often in the context of other patches which updates tests, so a larger number of discussions can be found in the archives. Reviewed-by: Julien Rouhaud <rjuju123@gmail.com> Reviewed-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com> Discussion: https://postgr.es/m/DD399313-3D56-4666-8079-88949DAC870F@yesql.se
2022-02-11 20:54:44 +01:00
done_testing();