postgresql/src
Heikki Linnakangas bde7493d10 Fix overflow check and comment in GIN posting list encoding.
The comment did not match what the code actually did for integers with
the 43rd bit set. You get an integer like that, if you have a posting
list with two adjacent TIDs that are more than 2^31 blocks apart.
According to the comment, we would store that in 6 bytes, with no
continuation bit on the 6th byte, but in reality, the code encodes it
using 7 bytes, with a continuation bit on the 6th byte as normal.

The decoding routine also handled these 7-byte integers correctly, except
for an overflow check that assumed that one integer needs at most 6 bytes.
Fix the overflow check, and fix the comment to match what the code
actually does. Also fix the comment that claimed that there are 17 unused
bits in the 64-bit representation of an item pointer. In reality, there
are 64-32-11=21.

Fitting any item pointer into max 6 bytes was an important property when
this was written, because in the old pre-9.4 format, item pointers were
stored as plain arrays, with 6 bytes for every item pointer. The maximum
of 6 bytes per integer in the new format guaranteed that we could convert
any page from the old format to the new format after upgrade, so that the
new format was never larger than the old format. But we hardly need to
worry about that anymore, and running into that problem during upgrade,
where an item pointer is expanded from 6 to 7 bytes such that the data
doesn't fit on a page anymore, is implausible in practice anyway.

Backpatch to all supported versions.

This also includes a little test module to test these large distances
between item pointers, without requiring a 16 TB table. It is not
backpatched, I'm including it more for the benefit of future development
of new posting list formats.

Discussion: https://www.postgresql.org/message-id/33bfc20a-5c86-f50c-f5a5-58e9925d05ff%40iki.fi
Reviewed-by: Masahiko Sawada, Alexander Korotkov
2019-08-28 12:55:33 +03:00
..
backend Fix overflow check and comment in GIN posting list encoding. 2019-08-28 12:55:33 +03:00
bin Disable timeouts when running pg_rewind with online source cluster 2019-08-28 11:47:35 +09:00
common Refactor logic to remove trailing CR/LF characters from strings 2019-08-09 11:05:14 +09:00
fe_utils Fix memory leak coming from simple lists built in reindexdb 2019-07-30 10:54:48 +09:00
include Avoid catalog lookups in RelationAllowsEarlyPruning(). 2019-08-28 16:18:29 +12:00
interfaces Set application_name per-test in isolation and ecpg tests. 2019-08-27 19:49:09 -04:00
makefiles Remove support for non-ELF BSD systems 2019-07-01 23:56:20 +01:00
pl Remove master/slave usage from plpgsql tests 2019-08-21 11:47:44 +02:00
port Improve what pg_strsignal prints if we haven't got strsignal(3). 2019-08-27 17:24:47 -04:00
template Yet further rethinking of build changes for macOS Mojave. 2018-11-02 18:54:00 -04:00
test Fix overflow check and comment in GIN posting list encoding. 2019-08-28 12:55:33 +03:00
timezone Use zic's new "-b slim" option to generate smaller timezone files. 2019-08-19 13:17:02 -04:00
tools Remove configure detection of crypt() 2019-08-21 21:36:54 +02:00
tutorial Phase 2 pgindent run for v12. 2019-05-22 13:04:48 -04:00
.gitignore
DEVELOPERS
Makefile Fix partial-build problems introduced by having more generated headers. 2018-04-09 16:42:10 -04:00
Makefile.global.in Fix inconsistencies and typos in the tree 2019-07-29 12:28:30 +09:00
Makefile.shlib Remove support for non-ELF BSD systems 2019-07-01 23:56:20 +01:00
nls-global.mk Move logging.h and logging.c from src/fe_utils/ to src/common/. 2019-05-14 14:20:10 -04:00