This function served to support having prebuilt files in the source
tree for vpath builds. This is no longer possible (since
721856ff24); all built files are now always in the build tree. The
invocations of this function are no longer required.
Up to now, our distribution tarballs have included a plain-text form
of the installation.sgml chapter. The rationale for that was that a
recipient might not have either ready internet access or HTML-viewing
tools; a theory that seems downright quaint today. Maintaining the
ability to generate this file is not without cost, because it puts
special requirements on installation.sgml that are often overlooked.
Moreover, we are moving in the direction of making our distribution
tarballs be pure git snapshots for traceability/reproducibility
reasons; including generated files doesn't fit into that plan.
Hence, let's just drop INSTALL and remove the infrastructure for
generating it. The top-level README will now recommend visiting
our website to see the installation instructions. As a useful
side-effect, we can get rid of README.git which has provoked
confusion.
Discussion: https://postgr.es/m/20231220114927.faccqqprmuyrzdip@alap3.anarazel.de
Discussion: https://postgr.es/m/e07408d9-e5f2-d9fd-5672-f53354e9305e@eisentraut.org
Currently important build targets are somewhat hard to discover. This commit
documents important meson build targets in the sgml documentation. But it's
awkward to have to lookup build targets in the docs when hacking, so this also
adds a 'help' target, printing out the same information. To avoid having to
duplicate information in two places, generate both docbook and interactive
docs from a single source.
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/20231108232121.ww542mt6lfo6f26f@awork3.anarazel.de
This undoes the change in what the 'docs' target builds 969509c3f2. Tom was
concerned with having a target to just build the html docs, which a prior
commit now provided explicitly.
A subsequent commit will overhaul the documentation for the documentation
targets.
While at it, move all target in doc/src/sgml/Makefile up to just after the
default "html" target, and add a comment explaining "all" is *not* the default
target.
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/20230209203855.njrepiupc3rmehfw@awork3.anarazel.de
Discussion: https://postgr.es/m/20231103163848.26egkh5qdgw3vmil@awork3.anarazel.de
A PostgreSQL release tarball contains a number of prebuilt files, in
particular files produced by bison, flex, perl, and well as html and
man documentation. We have done this consistent with established
practice at the time to not require these tools for building from a
tarball. Some of these tools were hard to get, or get the right
version of, from time to time, and shipping the prebuilt output was a
convenience to users.
Now this has at least two problems:
One, we have to make the build system(s) work in two modes: Building
from a git checkout and building from a tarball. This is pretty
complicated, but it works so far for autoconf/make. It does not
currently work for meson; you can currently only build with meson from
a git checkout. Making meson builds work from a tarball seems very
difficult or impossible. One particular problem is that since meson
requires a separate build directory, we cannot make the build update
files like gram.h in the source tree. So if you were to build from a
tarball and update gram.y, you will have a gram.h in the source tree
and one in the build tree, but the way things work is that the
compiler will always use the one in the source tree. So you cannot,
for example, make any gram.y changes when building from a tarball.
This seems impossible to fix in a non-horrible way.
Second, there is increased interest nowadays in precisely tracking the
origin of software. We can reasonably track contributions into the
git tree, and users can reasonably track the path from a tarball to
packages and downloads and installs. But what happens between the git
tree and the tarball is obscure and in some cases non-reproducible.
The solution for both of these issues is to get rid of the step that
adds prebuilt files to the tarball. The tarball now only contains
what is in the git tree (*). Getting the additional build
dependencies is no longer a problem nowadays, and the complications to
keep these dual build modes working are significant. And of course we
want to get the meson build system working universally.
This commit removes the make distprep target altogether. The make
dist target continues to do its job, it just doesn't call distprep
anymore.
(*) - The tarball also contains the INSTALL file that is built at make
dist time, but not by distprep. This is unchanged for now.
The make maintainer-clean target, whose job it is to remove the
prebuilt files in addition to what make distclean does, is now just an
alias to make distprep. (In practice, it is probably obsolete given
that git clean is available.)
The following programs are now hard build requirements in configure
(they were already required by meson.build):
- bison
- flex
- perl
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/flat/e07408d9-e5f2-d9fd-5672-f53354e9305e@eisentraut.org
The documentation and the code is generated automatically from a new
file called wait_event_names.txt, formatted in sections dedicated to
each wait event class (Timeout, Lock, IO, etc.) with three tab-separated
fields:
- C symbol in enums
- Format in the system views
- Description in the docs
Using this approach has several advantages, as we have proved to be
rather bad in maintaining this area of the tree across the years:
- The order of each item in the documentation and the code, which should
be alphabetical, has become incorrect multiple times, and the script
generating the code and documentation has a few rules to enforce that,
making the maintenance a no-brainer.
- Some wait events were added to the code, but not documented, so this
cannot be missed now.
- The order of the tables for each wait event class is enforced in the
documentation (the input .txt file does so as well for clarity, though
this is not mandatory).
- Less code, shaving 1.2k lines from the tree, with 1/3 of the savings
coming from the code, the rest from the documentation.
The wait event types "Lock" and "LWLock" still have their own code path
for their code, hence only the documentation is created for them. These
classes are listed with a special marker called WAIT_EVENT_DOCONLY in
the input file.
Adding a new wait event now requires only an update of
wait_event_names.txt, with "Lock" and "LWLock" treated as exceptions.
This commit has been tested with configure/Makefile, the CI and VPATH
build. clean, distclean and maintainer-clean were working fine.
Author: Bertrand Drouvot, Michael Paquier
Discussion: https://postgr.es/m/77a86b3a-c4a8-5f5d-69b9-d70bbf2e9b98@gmail.com
Until now the meson built docs did not have a working reference to the css
stylesheet, it was copied in the make target. Instead of duplicating that for
meson, use the docbook-xsl parameter custom.css.source to reference it. An
additional benefit of that approach is that the stylesheet is now included in
the single-file HTML documentation.
Reported-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/3fc3bb9b-f7f8-d442-35c1-ec82280c564a@enterprisedb.com
Until now the meson built HTML docs had non-working references to images. They
were copied in the make target. Instead of duplicating that for meson, copy
them as part of the xslt stylesheet.
Reported-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/3fc3bb9b-f7f8-d442-35c1-ec82280c564a@enterprisedb.com
We had partial support for generating documentation suitable for .chm
files. However, we only had wired up generating the input files using
docbook-xsl, not generating an actual .chm file. Nor did we document how to do
so. Additionally, it was very slow to generate htmlhelp, as we never applied
the docbook-xsl stylesheet performance improvements to htmlhelp.
It doesn't look like there's any interest in the htmlhelp output, so remove
it, instead of spending cycles to finish the support.
Discussion: https://postgr.es/m/20230324165822.wcrj3akllbqquy7u@awork3.anarazel.de
It appears no longer possible to build the SGML docs without a local
installation of the DocBook DTD, because sourceforge.net now only
permits HTTPS access, and no common version of xsltproc supports that.
Hence, remove the bits of our documentation suggesting that that's
possible or useful.
In fact, we might as well add the --nonet option to the build recipes
automatically, for a bit of extra security.
Also fix our documentation-tool-installation recipes for macOS to
ensure that xmllint and xsltproc are pulled in from MacPorts or
Homebrew. The previous recipes assumed you could use the
Apple-supplied versions of these tools; which still works, except that
you'd need to set an environment variable to ensure that they would
find DTD files provided by those package managers. Simpler and easier
to just recommend pulling in the additional packages.
In HEAD, also document how to build docs using Meson, and adjust
"ninja docs" to just build the HTML docs, for consistency with the
default behavior of doc/src/sgml/Makefile.
In a fit of neatnik-ism, I also made the ordering of the package
lists match the order in which the tools are described at the head
of the appendix.
Aleksander Alekseev, Peter Eisentraut, Tom Lane
Discussion: https://postgr.es/m/CAJ7c6TO8Aro2nxg=EQsVGiSDe-TstP4EsSvDHd7DSRsP40PgGA@mail.gmail.com
Before, each documentation target that built something from
postgres.sgml ran xmllint first to validate the input. Here, we
change it so that the validation only runs once and produces an output
file, and all the other targets build from that output file. This
avoids redundant work when building multiple documentation targets
(such as html and man).
Also, when we run xmllint, we can resolve entities (included files).
This helps with tools that don't support vpath builds, such as
dbtoepub.
All this also organizes the make targets a bit better for implementing
equivalent steps in meson.
Discussion: https://www.postgresql.org/message-id/e3ae16de-c9f9-f559-2d11-70b1342ae3d1@enterprisedb.com
Since the whole documentation tool chain is now UTF-8 and there is an
increasing number of non-ISO-8859-1 characters in the text, keeping
the Texinfo output in ISO 8859-1 just creates unnecessary
complications. Depending on the platform, there are conversion
failures and thus build failures, or weirdly converted characters. By
changing the output to UTF-8, the whole encoding conversion business
is sidestepped.
The SQL keywords table in the documentation had until now been
generated by some ad hoc scripting outside the source tree once for
each major release. This changes it to an automated process.
We have the PostgreSQL keywords available in a parseable format in
parser/kwlist.h. For the relevant SQL standard versions, keep the
keyword lists in new text files. A new script
generate-keywords-table.pl pulls it all together and produces a
DocBook table.
The final output in the documentation should be identical after this
change.
Discussion: https://www.postgresql.org/message-id/flat/07daeadd-8c82-0d95-5e19-e350502cb749%402ndquadrant.com
The makefile rule for the (rarely used) plain-text output postgres.txt
was still written to use lynx, but in
96b8b8b6f9, where the INSTALL file was
switched to pandoc, the rest of the makefile support for lynx was
removed, so this was broken. Rewrite the rule to also use pandoc for
postgres.txt.
In previous releases, the input file of dbtoepub was postgres.xml, and
dbtoepub knows to derive the output file name postgres.epub from that
automatically. But now the intput file is postgres.sgml (since
postgres.sgml is itself an XML file and we no longer need the
intermediate postgres.xml file), but dbtoepub doesn't know how to deal
with the .sgml suffix, so the automatically derived output file name
becomes postgres.sgml.epub. Fix by adding an explicit -o option.
With the previous rule, if pandoc was missing, a zero-length output
file would be created without an error from make. To improve that,
write the rule as two separate commands without a pipe.
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Commit 372728b0d created some problems for usages like building a
subdirectory without having first done "make all" at the top level,
or for proceeding directly to "make install" without "make all".
The only reasonably clean way to fix this seems to be to force the
submake-generated-headers rule to fire in *any* "make all" or "make
install" command anywhere in the tree. To avoid lots of redundant work,
as well as parallel make jobs possibly clobbering each others' output, we
still need to be sure that the rule fires only once in a recursive build.
For that, adopt the same MAKELEVEL hack previously used for "temp-install".
But try to document it a bit better.
The submake-errcodes mechanism previously used in src/port/ and src/common/
is subsumed by this, so we can get rid of those special cases. It was
inadequate for src/common/ anyway after the aforesaid commit, and it always
risked parallel attempts to build errcodes.h.
Discussion: https://postgr.es/m/E1f5FAB-0006LU-MB@gemulon.postgresql.org
Since some preparation work had already been done, the only source
changes left were changing empty-element tags like <xref linkend="foo">
to <xref linkend="foo"/>, and changing the DOCTYPE.
The source files are still named *.sgml, but they are actually XML files
now. Renaming could be considered later.
In the build system, the intermediate step to convert from SGML to XML
is removed. Everything is build straight from the source files again.
The OpenSP (or the old SP) package is no longer needed.
The documentation toolchain instructions are updated and are much
simpler now.
Peter Eisentraut, Alexander Lakhin, Jürgen Purtz
For DocBook XML compatibility, don't use SGML empty tags (</>) anymore,
replace by the full tag name. Add a warning option to catch future
occurrences.
Alexander Lakhin, Jürgen Purtz
For XML compatibility, replace marked sections <![IGNORE[ ]]> with
comments <!-- -->. In some cases it seemed better to remove the ignored
text altogether, and in one case the text should not have been ignored.
Previously, we created a variant of the installation instructions for
producing the plain-text INSTALL file by marking up certain parts of
installation.sgml using SGML parameterized marked sections. Marked
sections will not work anymore in XML, so before we can convert the
documentation to XML, we need a new approach.
DocBook provides a "profiling" feature that allows selecting content
based on attributes, which would work here. But it imposes a noticeable
overhead when building the full documentation and causes complications
when building some output formats, and given that we recently spent a
fair amount of effort optimizing the documentation build time, it seems
sad to have to accept that.
So as an alternative, (1) we create our own mini-profiling layer that
adjusts just the text we want, and (2) assemble the pieces of content
that we want in the INSTALL file using XInclude. That way, there is no
overhead when building the full documentation and most of the "ugly"
stuff in installation.sgml can be removed and dealt with out of line.
This is not required in SGML, but will be in XML, so this is a step to
prepare for the conversion to XML. (It is still not required to escape
>, but we did it here in some cases for symmetry.)
Add a command-line option to osx/onsgmls calls to warn about unescaped
occurrences in the future.
Author: Alexander Law <exclusion@gmail.com>
Author: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
This enables the same OpenSP warnings on osx calls that we get from
onsgmls (make check) and formerly from openjade.
Older tool chains apparently have some of these warnings on by
default (see comment at SPFLAGS assignment). So users of such tool
chains would complain about warnings or errors that users of newer tool
chains would not see, unless they used "make check".
All documentation is now built using XSLT. Remove all references to
Jade, DSSSL, also JadeTex and some other outdated tooling.
For chunked HTML builds, this changes nothing, but removes the
transitional "oldhtml" target. The single-page HTML build is ported
over to XSLT. For PDF builds, this removes the JadeTex builds and moves
the FOP builds in their place.
As before, create an INSTALL.html file for processing with lynx, but use
xsltproc and a new XSLT stylesheet instead of jade and DSSSL.
Replacing jade with xsltproc removes jade from the requirements for
distribution building.
Reviewed-by: Magnus Hagander <magnus@hagander.net>
SCO OpenServer and SCO UnixWare are more or less dead platforms.
We have never had a buildfarm member testing the "sco" port, and
the last "unixware" member was last heard from in 2012, so it's
fair to doubt that the code even compiles anymore on either one.
Remove both ports. We can always undo this if someone shows up
with an interest in maintaining and testing these platforms.
Discussion: <17177.1476136994@sss.pgh.pa.us>
Building the documentation with XSLT does not check the DTD, like a
DSSSL build would. One can often get away with having invalid XML, but
the stylesheets might then create incorrect output, as they are not
designed to handle that. Therefore, check the validity of the XML
against the DTD, using xmllint, during the build.
Add xmllint detection to configure, and add some documentation.
xmllint comes with libxml2, which is already in use, but it might be in
a separate package, such as libxml2-utils on Debian.
Reviewed-by: Fabien COELHO <coelho@cri.ensmp.fr>
Instead of just erroring out when a tool is missing, wrap the call with
the "missing" script that we are already using for bison, flex, and
perl, so that the users get a useful error message.
Providing this information as plain text was doubtless worth the trouble
ten years ago, but it seems likely that hardly anyone reads it in this
format anymore. And the effort required to maintain these files (in the
form of extra-complex markup rules in the relevant parts of the SGML
documentation) is significant. So, let's stop doing that and rely solely
on the other documentation formats.
Per discussion, the plain-text INSTALL instructions might still be worth
their keep, so we continue to generate that file.
Rather than remove HISTORY and src/test/regress/README from distribution
tarballs entirely, replace them with simple stub files that tell the reader
where to find the relevant documentation. This is mainly to avoid possibly
breaking packaging recipes that expect these files to exist.
Back-patch to all supported branches, because simplifying the markup
requirements for release notes won't help much unless we do it in all
branches.
Add a makefile rule for building PDFs with FOP. Two new build targets
in doc/src/sgml are postgres-A4-fop.pdf and postgres-US-fop.pdf.
Run .fo output through xmllint for reformatting, so that errors are
easier to find. (The default output has hardly any line breaks, so you
might be looking for an error in column 20000.)
Set some XSLT parameters to optimize for building with FOP.
Remove some redundant or somewhat useless chapterinfo/author
information, because it renders strangely with the FO stylesheet.
Reviewed-by: Álvaro Herrera <alvherre@2ndquadrant.com>
The XSLT toolchain requires an empty <index> element where the index is
supposed to appear. Add that with conditionals to hide it from the
DSSSL build.
The previous plan of having the check-tabs target a prerequisite of
"all" and "distprep" caused make distcheck to fail because make -q
distprep would never be satisfied. Put check-tabs into the html target
instead, so it is only called when a build actually happens.
make maintainer-check was obscure and rarely called in practice, and
many breakages were missed. Fold everything that make maintainer-check
used to do into the normal build. Specifically:
- Call duplicate_oids when genbki.pl is called.
- Check for tabs in SGML files when the documentation is built.
- Run msgfmt with the -c option during the regular build. Add an
additional configure check to see whether we are using the GNU
version. (make maintainer-check probably used to fail with non-GNU
msgfmt.)
Keep maintainer-check as around as phony target for the time being in
case anyone is calling it. But it won't do anything anymore.
This way it works more like the DSSSL build, and dependencies are
tracked better by make.
Also copy the CSS stylesheet to the html directory. This was forgotten
when the output directory was changed.
Peter Eisentraut
Tom Lane
Andres Freund
Michael Paquier
Peter Eisentraut
Bruce Momjian