Tomas Vondra f8397c955e Do not decode TOAST data for table rewrites ... During table rewrites (VACUUM FULL and CLUSTER), the main heap is logged using XLOG / FPI records, and thus (correctly) ignored in decoding. But the associated TOAST table is WAL-logged as plain INSERT records, and so was logically decoded and passed to reorder buffer. That has severe consequences with TOAST tables of non-trivial size. Firstly, reorder buffer has to keep all those changes, possibly spilling them to a file, incurring I/O costs and disk space. Secondly, ReoderBufferCommit() was stashing all those TOAST chunks into a hash table, which got discarded only after processing the row from the main heap. But as the main heap is not decoded for rewrites, this never happened, so all the TOAST data accumulated in memory, resulting either in excessive memory consumption or OOM. The fix is simple, as commit e9edc1ba already introduced infrastructure (namely HEAP_INSERT_NO_LOGICAL flag) to skip logical decoding of TOAST tables, but it only applied it to system tables. So simply use it for all TOAST data in raw_heap_insert(). That would however solve only the memory consumption issue - the TOAST changes would still be decoded and added to the reorder buffer, and spilled to disk (although without TOAST tuple data, so much smaller). But we can solve that by tweaking DecodeInsert() to just ignore such INSERT records altogether, using XLH_INSERT_CONTAINS_NEW_TUPLE flag, instead of skipping them later in ReorderBufferCommit(). Review: Masahiko Sawada Discussion: https://www.postgresql.org/message-id/flat/1a17c643-e9af-3dba-486b-fbe31bc1823a%402ndquadrant.com Backpatch: 9.4-, where logical decoding was introduced		2018-11-28 01:43:53 +01:00
..
backend	Do not decode TOAST data for table rewrites	2018-11-28 01:43:53 +01:00
bin	Fix ac218aa4f6 to work on versions before 9.5.	2018-11-26 23:26:15 -08:00
common	Revert "Allow concurrent-safe open() and fopen() in frontend code for Windows"	2018-09-18 17:28:27 -04:00
fe_utils	Fix translation of special characters in psql's LaTeX output modes.	2018-11-26 17:32:51 -05:00
include	PANIC on fsync() failure.	2018-11-19 13:37:59 +13:00
interfaces	Second try at fixing numeric data passed through an ECPG SQLDA.	2018-11-14 11:27:30 -05:00
makefiles	Refactor installation of extension headers.	2018-09-07 14:30:15 +01:00
pl	Fix error-cleanup mistakes in exec_stmt_call().	2018-11-09 22:04:14 -05:00
port	Stamp 11.1.	2018-11-05 16:43:09 -05:00
template	Yet further rethinking of build changes for macOS Mojave.	2018-11-02 18:54:00 -04:00
test	Fix jit compilation bug on wide tables.	2018-11-27 10:07:43 -08:00
timezone	Sync our copy of the timezone library with IANA release tzcode2018g.	2018-10-31 09:48:04 -04:00
tools	Add valgrind suppressions for wcsrtombs optimizations	2018-11-18 00:08:19 +01:00
tutorial	Update copyright for 2018	2018-01-02 23:30:12 -05:00
.gitignore
DEVELOPERS
Makefile	Fix partial-build problems introduced by having more generated headers.	2018-04-09 16:42:10 -04:00
Makefile.global.in	Yet further rethinking of build changes for macOS Mojave.	2018-11-02 18:54:00 -04:00
Makefile.shlib	Use -Bsymbolic for shared libraries on HP-UX and Solaris.	2018-09-10 22:22:12 -04:00
nls-global.mk	nls-global.mk: search build dir for source files, too	2016-06-07 18:55:18 -04:00