rdelaage/postgresql
1
0
Fork 0
mirror of https://git.postgresql.org/git/postgresql.git synced 2024-09-09 00:19:23 +02:00
Code Issues Packages Projects Releases Wiki Activity
51,004 Commits 36 Branches 606 Tags 423 MiB
C 86%
PLpgSQL 5.5%
Perl 4.1%
Yacc 1.3%
Makefile 0.7%
Other 2.3%
6606c57162
Go to file
Thomas Munro 6606c57162 Don't trust unvalidated xl_tot_len. 
xl_tot_len comes first in a WAL record.  Usually we don't trust it to be
the true length until we've validated the record header.  If the record
header was split across two pages, previously we wouldn't do the
validation until after we'd already tried to allocate enough memory to
hold the record, which was bad because it might actually be garbage
bytes from a recycled WAL file, so we could try to allocate a lot of
memory.  Release 15 made it worse.

Since 70b4f82a4b, we'd at least generate an end-of-WAL condition if the
garbage 4 byte value happened to be > 1GB, but we'd still try to
allocate up to 1GB of memory bogusly otherwise.  That was an
improvement, but unfortunately release 15 tries to allocate another
object before that, so you could get a FATAL error and recovery could
fail.

We can fix both variants of the problem more fundamentally using
pre-existing page-level validation, if we just re-order some logic.

The new order of operations in the split-header case defers all memory
allocation based on xl_tot_len until we've read the following page.  At
that point we know that its first few bytes are not recycled data, by
checking its xlp_pageaddr, and that its xlp_rem_len agrees with
xl_tot_len on the preceding page.  That is strong evidence that
xl_tot_len was truly the start of a record that was logged.

This problem was most likely to occur on a standby, because
walreceiver.c recycles WAL files without zeroing out trailing regions of
each page.  We could fix that too, but it wouldn't protect us from rare
crash scenarios where the trailing zeroes don't make it to disk.

With reliable xl_tot_len validation in place, the ancient policy of
considering malloc failure to indicate corruption at end-of-WAL seems
quite surprising, but changing that is left for later work.

Also included is a new TAP test to exercise various cases of end-of-WAL
detection by writing contrived data into the WAL from Perl.

Back-patch to 12.  We decided not to put this change into the final
release of 11.

Author: Thomas Munro <thomas.munro@gmail.com>
Author: Michael Paquier <michael@paquier.xyz>
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Reviewed-by: Noah Misch <noah@leadboat.com> (the idea, not the code)
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Sergei Kornilov <sk@zsrv.org>
Reviewed-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/17928-aa92416a70ff44a2%40postgresql.org
2023-09-23 10:29:44 +12:00
config Use --strip-unneeded when stripping static libraries with GNU strip. 2023-04-20 18:12:32 -04:00
contrib postgres_fdw: Fix test for parameterized foreign scan. 2023-08-30 17:15:07 +09:00
doc Doc: copy-edit the introductory para for the pg_class catalog. 2023-09-22 14:52:36 -04:00
src Don't trust unvalidated xl_tot_len. 2023-09-23 10:29:44 +12:00
.dir-locals.el Make Emacs perl-mode indent more like perltidy. 2019-01-13 11:32:31 -08:00
.editorconfig Add .editorconfig 2019-12-18 09:13:13 +01:00
.git-blame-ignore-revs Backpatch addition of .git-blame-ignore-revs 2022-08-05 19:36:24 +02:00
.gitattributes gitattributes: Add new file 2019-11-12 08:13:55 +01:00
.gitignore Add portlock directory to .gitignore 2022-11-26 07:47:16 -05:00
aclocal.m4 Probe $PROVE not $PERL while checking for modules needed by TAP tests. 2021-11-22 12:54:52 -05:00
configure Stamp 13.12. 2023-08-07 16:11:34 -04:00
configure.in Stamp 13.12. 2023-08-07 16:11:34 -04:00
COPYRIGHT Update copyright for 2023 2023-01-02 15:00:36 -05:00
GNUmakefile.in Remove temporary portlock directory during make [dist]clean. 2022-11-26 10:30:58 -05:00
HISTORY Canonicalize some URLs 2020-02-10 20:47:50 +01:00
Makefile Don't unset MAKEFLAGS in non-GNU Makefile. 2019-06-25 09:36:21 +12:00
README Canonicalize some URLs 2020-02-10 20:47:50 +01:00
README.git Canonicalize some URLs 2020-02-10 20:47:50 +01:00

README 

PostgreSQL Database Management System
=====================================

This directory contains the source code distribution of the PostgreSQL
database management system.

PostgreSQL is an advanced object-relational database management system
that supports an extended subset of the SQL standard, including
transactions, foreign keys, subqueries, triggers, user-defined types
and functions.  This distribution also contains C language bindings.

PostgreSQL has many language interfaces, many of which are listed here:

	https://www.postgresql.org/download/

See the file INSTALL for instructions on how to build and install
PostgreSQL.  That file also lists supported operating systems and
hardware platforms and contains information regarding any other
software packages that are required to build or run the PostgreSQL
system.  Copyright and license information can be found in the
file COPYRIGHT.  A comprehensive documentation set is included in this
distribution; it can be read as described in the installation
instructions.

The latest version of this software may be obtained at
https://www.postgresql.org/download/.  For more information look at our
web site located at https://www.postgresql.org/.