postgresql/src/test/perl
Tom Lane 3911fd55f5 Create common infrastructure for cross-version upgrade testing.
To test pg_upgrade across major PG versions, we have to be able to
modify or drop any old objects with no-longer-supported properties,
and we have to be able to deal with cosmetic changes in pg_dump output.
Up to now, the buildfarm and pg_upgrade's own test infrastructure had
separate implementations of the former, and we had nothing but very
ad-hoc rules for the latter (including an arbitrary threshold on how
many lines of unchecked diff were okay!).  This patch creates a Perl
module that can be shared by both those use-cases, and adds logic
that deals with pg_dump output diffs in a much more tightly defined
fashion.

This largely supersedes previous efforts in commits 0df9641d3,
9814ff550, and 62be9e4cd, which developed a SQL-script-based solution
for the task of dropping old objects.  There was nothing fundamentally
wrong with that work in itself, but it had no basis for solving the
output-formatting problem.  The most plausible way to deal with
formatting is to build a Perl module that can perform editing on the
dump files; and once we commit to that, it makes more sense for the
same module to also embed the knowledge of what has to be done for
dropping old objects.

Back-patch versions of the helper module as far as 9.2, to
support buildfarm animals that still test that far back.
It's also necessary to back-patch PostgreSQL/Version.pm,
because the new code depends on that.  I fixed up pg_upgrade's
002_pg_upgrade.pl in v15, but did not look into back-patching
it further than that.

Tom Lane and Andrew Dunstan

Discussion: https://postgr.es/m/891521.1673657296@sss.pgh.pa.us
2023-01-16 20:35:53 -05:00
..
PostgreSQL Create common infrastructure for cross-version upgrade testing. 2023-01-16 20:35:53 -05:00
Makefile Install TAP test infrastructure so it's available for extension testing. 2016-09-23 15:50:00 -04:00
PostgresNode.pm Fix CREATE INDEX CONCURRENTLY for the newest prepared transactions. 2021-10-23 18:36:43 -07:00
README Make PostgresNode easily subclassable 2017-07-25 18:50:53 -04:00
RecursiveCopy.pm Fix RecursiveCopy.pm to cope with disappearing files. 2017-09-11 22:02:58 -04:00
SimpleTee.pm perltidy PostgresNode and SimpleTee 2016-03-03 13:21:35 -03:00
TestLib.pm Account for IPC::Run::result() Windows behavior change. 2022-11-17 07:43:14 -08:00

README

Perl-based TAP tests
====================

src/test/perl/ contains shared infrastructure that's used by Perl-based tests
across the source tree, particularly tests in src/bin and src/test. It's used
to drive tests for backup and restore, replication, etc - anything that can't
really be expressed using pg_regress or the isolation test framework.

You should prefer to write tests using pg_regress in src/test/regress, or
isolation tester specs in src/test/isolation, if possible. If not, check to
see if your new tests make sense under an existing tree in src/test, like
src/test/ssl, or should be added to one of the suites for an existing utility.

Note that all tests and test tools should have perltidy run on them before
patches are submitted, using perltidy --profile=src/tools/pgindent/perltidyrc

By default, to keep the noise low during runs, we do not set any flags via
PROVE_FLAGS, but this can be done on the 'make' command line if desired, eg:

make check-world PROVE_FLAGS='--verbose'

Writing tests
-------------

Tests are written using Perl's Test::More with some PostgreSQL-specific
infrastructure from src/test/perl providing node management, support for
invoking 'psql' to run queries and get results, etc. You should read the
documentation for Test::More before trying to write tests.

Test scripts in the t/ subdirectory of a suite are executed in alphabetical
order.

Each test script should begin with:

    use strict;
    use warnings;
    use PostgresNode;
    use TestLib;
    # Replace with the number of tests to execute:
    use Test::More tests => 1;

then it will generally need to set up one or more nodes, run commands
against them and evaluate the results. For example:

    my $node = PostgresNode->get_new_node('master');
    $node->init;
    $node->start;

    my $ret = $node->safe_psql('postgres', 'SELECT 1');
    is($ret, '1', 'SELECT 1 returns 1');

    $node->stop('fast');

Test::More::like entails use of the qr// operator.  Avoid Perl 5.8.8 bug
#39185 by not using the "$" regular expression metacharacter in qr// when also
using the "/m" modifier.  Instead of "$", use "\n" or "(?=\n|\z)".

Read the Test::More documentation for more on how to write tests:

    perldoc Test::More

For available PostgreSQL-specific test methods and some example tests read the
perldoc for the test modules, e.g.:

    perldoc src/test/perl/PostgresNode.pm