Commit Graph

94 Commits

Author SHA1 Message Date
Tom Lane
01df147634 Allow selecting the git revision to be packaged by "make dist".
Commit 619bc23a1 changed "make dist" to invoke "git archive",
but hard-wired the call to specify that the HEAD revision should
be packaged.  Our tarball building process needs to be able to
specify which git commit to package (notably, for packaging
back branches).  While we could make that work with some hackery
to operate in detached-HEAD state, it's a lot nicer just to expose
git archive's ability to specify what to package.  Hence, invent
a new make variable PG_GIT_REVISION.  This is undocumented, but
so is "make dist".

Also make corresponding changes in the meson scripts.  We have no
near-term intention of using that for package building, but it
will likely happen eventually, so stay prepared.

Discussion: https://postgr.es/m/3552543.1713909947@sss.pgh.pa.us
2024-05-03 11:08:50 -04:00
Peter Eisentraut
619bc23a1a make dist uses git archive
This changes "make dist" to directly use "git archive", rather than
the custom shell script it currently runs.

This is to make the creation of the distribution tarball more directly
traceable to the git repository.  That is why we removed the "make
distprep" step.

"make dist" continues to produce a .gz and a .bz2 tarball as before.

The archives produced this way are deterministic and reproducible,
meaning for a given commit the result file should always be
bit-for-bit identical.  The exception is that if you use a git version
older than 2.38.0, gzip records the platform in the archive, so you'd
get a different output on Windows vs. macOS vs. "UNIX" (everything
else).  In git 2.38.0, this was changed so that everything is recorded
as "UNIX" now.  This is just something to keep in mind.  This issue is
specific to the gzip format, it does not affect other compression
formats.

Meson has its own distribution building command (meson dist), but we
are not using that at this point.  The main problem is that, the way
they have implemented it, it is not deterministic in the above sense.
Also, we want a "make" version for the time being.  But the target
name "dist" in meson is reserved for that reason, so we call the
custom target "pgdist" (so call something like "meson compile -C build
pgdist").

Reviewed-by: Tristan Partin <tristan@neon.tech>
Discussion: https://www.postgresql.org/message-id/flat/40e80f77-a294-4f29-a16f-e21bc7bc75fc%40eisentraut.org
2024-03-25 06:29:53 +01:00
Peter Eisentraut
7b8e2ae2fd Combine headerscheck and cpluspluscheck scripts
They are mostly the same, and it is tedious to maintain two copies of
essentially the same exclude list.  headerscheck now has a new option
--cplusplus to select the cpluspluscheck functionality.  The top-level
make targets are still the same.

Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://www.postgresql.org/message-id/flat/4754a5b0-a32b-4036-a99a-6de14cf9fd72@eisentraut.org
2024-03-10 07:56:17 +01:00
Tom Lane
e2b73f4a4d Stop generating plain-text INSTALL instructions.
Up to now, our distribution tarballs have included a plain-text form
of the installation.sgml chapter.  The rationale for that was that a
recipient might not have either ready internet access or HTML-viewing
tools; a theory that seems downright quaint today.  Maintaining the
ability to generate this file is not without cost, because it puts
special requirements on installation.sgml that are often overlooked.
Moreover, we are moving in the direction of making our distribution
tarballs be pure git snapshots for traceability/reproducibility
reasons; including generated files doesn't fit into that plan.
Hence, let's just drop INSTALL and remove the infrastructure for
generating it.  The top-level README will now recommend visiting
our website to see the installation instructions.  As a useful
side-effect, we can get rid of README.git which has provoked
confusion.

Discussion: https://postgr.es/m/20231220114927.faccqqprmuyrzdip@alap3.anarazel.de
Discussion: https://postgr.es/m/e07408d9-e5f2-d9fd-5672-f53354e9305e@eisentraut.org
2023-12-22 13:32:15 -05:00
Peter Eisentraut
721856ff24 Remove distprep
A PostgreSQL release tarball contains a number of prebuilt files, in
particular files produced by bison, flex, perl, and well as html and
man documentation.  We have done this consistent with established
practice at the time to not require these tools for building from a
tarball.  Some of these tools were hard to get, or get the right
version of, from time to time, and shipping the prebuilt output was a
convenience to users.

Now this has at least two problems:

One, we have to make the build system(s) work in two modes: Building
from a git checkout and building from a tarball.  This is pretty
complicated, but it works so far for autoconf/make.  It does not
currently work for meson; you can currently only build with meson from
a git checkout.  Making meson builds work from a tarball seems very
difficult or impossible.  One particular problem is that since meson
requires a separate build directory, we cannot make the build update
files like gram.h in the source tree.  So if you were to build from a
tarball and update gram.y, you will have a gram.h in the source tree
and one in the build tree, but the way things work is that the
compiler will always use the one in the source tree.  So you cannot,
for example, make any gram.y changes when building from a tarball.
This seems impossible to fix in a non-horrible way.

Second, there is increased interest nowadays in precisely tracking the
origin of software.  We can reasonably track contributions into the
git tree, and users can reasonably track the path from a tarball to
packages and downloads and installs.  But what happens between the git
tree and the tarball is obscure and in some cases non-reproducible.

The solution for both of these issues is to get rid of the step that
adds prebuilt files to the tarball.  The tarball now only contains
what is in the git tree (*).  Getting the additional build
dependencies is no longer a problem nowadays, and the complications to
keep these dual build modes working are significant.  And of course we
want to get the meson build system working universally.

This commit removes the make distprep target altogether.  The make
dist target continues to do its job, it just doesn't call distprep
anymore.

(*) - The tarball also contains the INSTALL file that is built at make
dist time, but not by distprep.  This is unchanged for now.

The make maintainer-clean target, whose job it is to remove the
prebuilt files in addition to what make distclean does, is now just an
alias to make distprep.  (In practice, it is probably obsolete given
that git clean is available.)

The following programs are now hard build requirements in configure
(they were already required by meson.build):

- bison
- flex
- perl

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/flat/e07408d9-e5f2-d9fd-5672-f53354e9305e@eisentraut.org
2023-11-06 15:18:04 +01:00
Tom Lane
156c049bee Integrate pg_bsd_indent into our build/test infrastructure.
Update the Makefile and build directions for in-tree build,
and add Meson build infrastructure.  Also convert the ad-hoc
test target into a TAP test.

Currently, the Make build system will not build pg_bsd_indent
by default, while the Meson system will.  Both will test it
during "make check-world" or "ninja test".  Neither will install
it automatically.  (We might change some of these decisions later.)

Also fix a few portability nits noted during early testing.

Also, exclude pg_bsd_indent from pgindent's purview; at least for
now, we'll leave it formatted similarly to the FreeBSD original.

Tom Lane and Andres Freund

Discussion: https://postgr.es/m/3935719.1675967430@sss.pgh.pa.us
Discussion: https://postgr.es/m/20200812223409.6di3y2qsnvynao7a@alap3.anarazel.de
2023-02-12 12:22:21 -05:00
Tom Lane
3274ec05c8 Remove temporary portlock directory during make [dist]clean.
Another oversight in 9b4eafcaf.
2022-11-26 10:30:31 -05:00
Michael Paquier
2cc96f2339 Run tests of libpq on installcheck-world, checkprep and check-world
The regression tests of libpq have been converted to a TAP test as of
ac25173, but they were not run when running the global make targets
mentioned in $subject as the code only considered src/interfaces/ecpg/
as a path to run.  This switches to src/interfaces/, to always include
the tests of libpq in each run.

Reported-by: Justin Pryzby
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/20220529151850.GN19626@telsasoft.com
2022-06-03 13:15:20 +09:00
Andrew Dunstan
7355c241ed
add missing tag from commit b8c4261e5e 2021-07-01 15:47:46 -04:00
Andrew Dunstan
b8c4261e5e
Add new make targets world-bin and install-world-bin
These are the same as world and install-world respectively, but without
building or installing the documentation. There are many reasons for
wanting to be able to do this, including speed, lack of documentation
building tools, and wanting to build other formats of the documentation.
Plans for simplifying the buildfarm client code include using these
targets.

Backpatch to all live branches.

Discussion: https://postgr.es/m/6a421136-d462-b043-a8eb-e75b2861f3df@dunslane.net
2021-07-01 15:12:21 -04:00
Andrew Dunstan
af4ea507c3 Make install-tests target work with vpath builds
Also add a top-level install-tests target.

Backpatch to all live branches.

Craig Ringer, tweaked by me.
2020-05-31 18:33:00 -04:00
Noah Misch
650eac8d7b Add a temp-install prerequisite to top-level "check-tests".
The target failed, tested $PATH binaries, or tested a stale temporary
installation.  Commit c66b438db6 missed
this.  Back-patch to 9.5 (all supported versions).
2020-05-25 16:21:04 -07:00
Peter Eisentraut
f85a485f89 Add support for automatically updating Unicode derived files
We currently have several sets of files generated from data provided
by Unicode.  These all have ad hoc rules and instructions for updating
when new Unicode versions appear, and it's not done consistently.

This patch centralizes and automates the process and makes it part of
the release checklist.  The Unicode and CLDR versions are specified in
Makefile.global.in.  There is a new make target "update-unicode" that
downloads all the relevant files and runs the generation script.

There is also a new script for generating the table of combining
characters for ucs_wcwidth().  That table is now in a separate include
file rather than hardcoded into the middle of other code.  This is
based on the script that was used for generating
d8594d123c, but the script itself wasn't
committed at that time.

Reviewed-by: John Naylor <john.naylor@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/c8d05f42-443e-6c23-819b-05b31759a37c@2ndquadrant.com
2020-01-09 10:08:14 +01:00
Tom Lane
55ea109188 Add "headerscheck" script to test header-file compilability under C.
We already had "cpluspluscheck", which served the dual purposes of
verifying that headers compile standalone and that they compile as C++.
However, C++ compilers don't have the exact same set of error conditions
as C compilers, so this doesn't really prove that a header will compile
standalone as C.

Hence, add a second script that's largely similar but runs the C
compiler not C++.

Also add a bit more documentation than the none-at-all we had before.

Discussion: https://postgr.es/m/14803.1566175851@sss.pgh.pa.us
2019-08-19 14:22:56 -04:00
Andres Freund
b1cd7ce23f Integrate cpluspluscheck into build system.
Previously cpluspluscheck wouldn't work in vpath builds, this commit
fixes that. To make it easier to invoke, there's a top-level
cpluspluscheck target.

Discussion: https://postgr.es/20190530220244.kiputcbl4gkl2oo6@alap3.anarazel.de
2019-05-31 12:36:17 -07:00
Noah Misch
aa019da523 Process EXTRA_INSTALL serially, during the first temp-install.
This closes a race condition in "make -j check-world"; the symptom was
EEXIST errors.  Back-patch to v10, before which parallel check-world had
worse problems.

Discussion: https://postgr.es/m/20181224221601.GA3227827@rfd.leadboat.com
2018-12-31 13:54:38 -08:00
Tom Lane
1eee8d4994 Remove unwanted "garbage cleanup" logic in Makefiles.
GNUmakefile.in defined a macro "garbage" that seems to have been meant
as a suitable target for automatic "rm -rf" treatment, but it isn't
actually used anywhere (and indeed never was, AFAICT).

Moreover, we have concluded that the Makefiles shouldn't take it upon
themselves to remove files that aren't expected by-products of building,
so that doing anything like that would be against project policy anyway.
Hence, just remove the macro.

Grepping around finds another violation of that policy in ecpg/preproc,
so clean that up too.

Daniel Gustafsson (ecpg change by me)

Discussion: https://postgr.es/m/AFBEF63E-E19D-4EBB-9F08-4617CDC751ED@yesql.se
2018-08-08 14:32:29 -04:00
Tom Lane
7dc5a96aa2 Ensure we build generated headers at the start of some more cases.
"make installcheck" and some related cases, when invoked from the toplevel
directory, start out by doing "make all" in src/test/regress.  Since that's
one make recursion level down, the submake-generated-headers target will
do nothing, causing us to fail to create/update generated headers before
building pg_regress.  This is, I believe, a new failure mode induced by
commit 3b8f6e75f, so let's fix it.  To do so, we have to invoke
submake-generated-headers at the top level.

Discussion: https://postgr.es/m/0401efec-68f1-679d-3ea3-21d4e8dd11af@gmail.com
2018-07-30 18:04:39 -04:00
Alvaro Herrera
2093f66305 Have "make coverage" recurse into contrib as well 2016-09-05 18:44:36 -03:00
Tom Lane
9660710e2f Fix distclean/maintainer-clean targets to remove top-level tmp_install dir.
The top-level makefile removes tmp_install in its "clean" target, but the
distclean and maintainer-clean targets overlooked that (and they don't
simply invoke clean, because that would result in an extra tree traversal).

While at it, let's just make sure that removing GNUmakefile itself is the
very last step of the recipe.
2015-05-13 18:48:05 -04:00
Peter Eisentraut
dbf2ec1a1c Fix parallel make risk with new check temp-install setup
The "check" target no longer needs to depend on "all", because it now
runs "install" directly, which in turn depends on "all".  Doing both
will cause problems with parallel make, because two builds will run next
to each other.

Also remove the redirection of the temp-install output into a log file.
This was appropriate when this was done from within pg_regress, but now
it's just a regular make run, and especially with the above changes this
will now take the place of running the "all" target before the test
suites.

problem report by Jeff Janes, patch in part by Michael Paquier
2015-04-29 20:34:22 -04:00
Peter Eisentraut
dcae5facca Improve speed of make check-world
Before, make check-world would create a new temporary installation for
each test suite, which is slow and wasteful.  Instead, we now create one
test installation that is used by all test suites that are part of a
make run.

The management of the temporary installation is removed from pg_regress
and handled in the makefiles.  This allows for better control, and
unifies the code with that of test suites not run through pg_regress.

review and msvc support by Michael Paquier <michael.paquier@gmail.com>

more review by Fabien Coelho <coelho@cri.ensmp.fr>
2015-04-23 08:59:52 -04:00
Peter Eisentraut
7d0f493f19 Add TAP tests for client programs
Reviewed-by: Pavel Stěhule <pavel.stehule@gmail.com>
Reviewed-by: Erik Rijkers <er@xs4all.nl>
2014-04-14 21:33:46 -04:00
Tom Lane
2895415205 Don't generate plain-text HISTORY and src/test/regress/README anymore.
Providing this information as plain text was doubtless worth the trouble
ten years ago, but it seems likely that hardly anyone reads it in this
format anymore.  And the effort required to maintain these files (in the
form of extra-complex markup rules in the relevant parts of the SGML
documentation) is significant.  So, let's stop doing that and rely solely
on the other documentation formats.

Per discussion, the plain-text INSTALL instructions might still be worth
their keep, so we continue to generate that file.

Rather than remove HISTORY and src/test/regress/README from distribution
tarballs entirely, replace them with simple stub files that tell the reader
where to find the relevant documentation.  This is mainly to avoid possibly
breaking packaging recipes that expect these files to exist.

Back-patch to all supported branches, because simplifying the markup
requirements for release notes won't help much unless we do it in all
branches.
2014-02-10 20:48:04 -05:00
Andrew Dunstan
7043ac7100 Add new make targets check-tests and installcheck-tests.
These do not run any specific schedule of tests, but only those
specified as part of the invocation, e.g.:

    make check-tests TESTS="json jsonb"
2014-01-28 18:10:00 -05:00
Peter Eisentraut
5dd41f3574 Remove maintainer-check target, fold into normal build
make maintainer-check was obscure and rarely called in practice, and
many breakages were missed.  Fold everything that make maintainer-check
used to do into the normal build.  Specifically:

- Call duplicate_oids when genbki.pl is called.

- Check for tabs in SGML files when the documentation is built.

- Run msgfmt with the -c option during the regular build.  Add an
  additional configure check to see whether we are using the GNU
  version.  (make maintainer-check probably used to fail with non-GNU
  msgfmt.)

Keep maintainer-check as around as phony target for the time being in
case anyone is calling it.  But it won't do anything anymore.
2013-10-10 20:11:56 -04:00
Peter Eisentraut
b344c651fb Make init-po and update-po recursive make targets
This is for convenience, now that adding recursive targets is much
easier than it used to be when the NLS stuff was initially added.
2012-06-29 14:01:54 +03:00
Peter Eisentraut
cecdf6d459 Add isolation test to check-world and installcheck-world 2012-03-05 20:19:20 +02:00
Peter Eisentraut
4803de6f89 Build src/ before contrib/ in make world
This fixes failures under parallel make when contrib modules use a
generated backend header file (such as errcodes.h).
2011-08-24 21:34:49 +03:00
Peter Eisentraut
f8ebe3bcc5 Support "make check" in contrib
Added a new option --extra-install to pg_regress to arrange installing
the respective contrib directory into the temporary installation.
This is currently not yet supported for Windows MSVC builds.

Updated the .gitignore files for contrib modules to ignore the
leftovers of a temp-install check run.

Changed the exit status of "make check" in a pgxs build (which still
does nothing) to 0 from 1.

Added "make check" in contrib to top-level "make check-world".
2011-04-25 22:27:11 +03:00
Peter Eisentraut
6c0dfc0356 Add maintainer-check target
This can do various source code checks that are not appropriate for
either the build or the regression tests.  Currently: duplicate_oids,
SGML syntax and tabs check, NLS syntax check.
2011-03-28 22:56:52 +03:00
Peter Eisentraut
2fd77060a2 Allow make check in PL directories
Also add make check-world target, and refactor pg_regress invocation
code in makefiles a bit.
2011-02-15 06:52:12 +02:00
Peter Eisentraut
19e231bbda Improved parallel make support
Replace for loops in makefiles with proper dependencies.  Parallel
make can now span across directories.  Also, make -k and make -q work
properly.

GNU make 3.80 or newer is now required.
2010-11-12 22:15:16 +02:00
Peter Eisentraut
dd21f0b047 Print a make warning when using GNU make older than 3.80
A proposed patch will require GNU make 3.80 or newer.  We will let this patch
run for a while to see how much damage that would do to the buildfarm.
2010-11-03 17:33:04 +02:00
Magnus Hagander
397761827a Rename README.CVS to README.git and change references in it. 2010-09-21 12:40:16 +02:00
Magnus Hagander
9f2e211386 Remove cvs keywords from all files. 2010-09-20 22:08:53 +02:00
Peter Eisentraut
29ccc32c30 Separate targets "make docs" and "make install-docs" for the documentation
It is no longer installed by default, but included in "make world"/"make
install-world".  Documentation updated accordingly.

Also, fix vpathsearch function to work when calling make install-docs
without previous make docs.
2010-03-30 00:10:46 +00:00
Andrew Dunstan
8d827f6b5c Fix thinko in new installcheck-world target 2010-01-29 01:06:18 +00:00
Andrew Dunstan
7523960d63 Add new make targets "world", "install-world" and "installcheck-world" to build, install and check just about everything.
In addition to everything built installed and tested by all, install and installcheck targets, these build HTML Docs,
build and test contrib, and test PLs and ECPG.
2010-01-28 23:59:52 +00:00
Peter Eisentraut
a4d03bbcda Build bzip2 tarball in dist target as well 2009-11-03 21:28:10 +00:00
Peter Eisentraut
90412225d2 Remove some leftovers of split tarball support 2009-10-29 21:57:17 +00:00
Alvaro Herrera
3a00c0a6d4 Be able to easily figure out the target directory of "make dist". 2009-08-14 21:37:18 +00:00
Peter Eisentraut
ed9208829a Ship documentation without intermediate tarballs
Documentation files in HTML and man formats are now prepared for
distribution using the distprep make target, like everything else.  They
are placed in doc/src/sgml/html and manX and installed from there by
make install, if present.  The business with the tarballs in the tarball
is gone.
2009-08-09 22:47:59 +00:00
Peter Eisentraut
7798147a76 Expand test coverage support to entire tree
Test coverage support now covers the entire source tree, including
contrib, instead of just src/backend.  In a related but independent
development, the commands make coverage and make coverage-html can be run
in any directory.

This turned out to be much easier than feared.  Besides a few ad hoc fixes
to pass the make target down the tree, change all affected makefiles to
list their directories in the SUBDIRS variable, changed from variants like
DIRS and WANTED_DIRS.  MSVC build fix was attempted as well.
2009-08-07 20:50:22 +00:00
Bruce Momjian
9173d759d2 Modify distdir rule to skip .git directory. 2009-01-15 01:53:49 +00:00
Peter Eisentraut
754e6833ee Call genhtml with --prefix option so local build paths don't appear in the
coverage output.
2008-12-04 07:02:56 +00:00
Peter Eisentraut
11f53b1063 Code coverage testing with gcov. Documentation is in the regression test
chapter.

Author: Michelle Caisse <Michelle.Caisse@Sun.COM>
2008-09-05 12:11:18 +00:00
Peter Eisentraut
8c87cc370f Catch all errors in for and while loops in makefiles. Don't ignore any
errors in any commands, including in various clean targets that have so far
been handled inconsistently.  make -i is available to ignore all errors in
a consistent and official way.
2008-03-18 16:24:50 +00:00
Peter Eisentraut
c138b966d4 Replace useless uses of := by = in makefiles. 2007-02-09 15:56:00 +00:00
Andrew Dunstan
1be439084a Add installcheck-parallel target to top level makefiles. 2006-08-18 19:58:05 +00:00