cec2edfa78
Instead of deciding to serialize a transaction merely based on the
number of changes in that xact (toplevel or subxact), this makes
the decisions based on amount of memory consumed by the changes.
The memory limit is defined by a new logical_decoding_work_mem GUC,
so for example we can do this
SET logical_decoding_work_mem = '128kB'
to reduce the memory usage of walsenders or set the higher value to
reduce disk writes. The minimum value is 64kB.
When adding a change to a transaction, we account for the size in
two places. Firstly, in the ReorderBuffer, which is then used to
decide if we reached the total memory limit. And secondly in the
transaction the change belongs to, so that we can pick the largest
transaction to evict (and serialize to disk).
We still use max_changes_in_memory when loading changes serialized
to disk. The trouble is we can't use the memory limit directly as
there might be multiple subxact serialized, we need to read all of
them but we don't know how many are there (and which subxact to
read first).
We do not serialize the ReorderBufferTXN entries, so if there is a
transaction with many subxacts, most memory may be in this type of
objects. Those records are not included in the memory accounting.
We also do not account for INTERNAL_TUPLECID changes, which are
kept in a separate list and not evicted from memory. Transactions
with many CTID changes may consume significant amounts of memory,
but we can't really do much about that.
The current eviction algorithm is very simple - the transaction is
picked merely by size, while it might be useful to also consider age
(LSN) of the changes for example. With the new Generational memory
allocator, evicting the oldest changes would make it more likely
the memory gets actually pfreed.
The logical_decoding_work_mem can be set in postgresql.conf, in which
case it serves as the default for all publishers on that instance.
Author: Tomas Vondra, with changes by Dilip Kumar and Amit Kapila
Reviewed-by: Dilip Kumar and Amit Kapila
Tested-By: Vignesh C
Discussion: https://postgr.es/m/688b0b7f-2f6c-d827-c27b-216a8e3ea700@2ndquadrant.com
|
||
|---|---|---|
| .. | ||
| adminpack | ||
| amcheck | ||
| auth_delay | ||
| auto_explain | ||
| bloom | ||
| btree_gin | ||
| btree_gist | ||
| citext | ||
| cube | ||
| dblink | ||
| dict_int | ||
| dict_xsyn | ||
| earthdistance | ||
| file_fdw | ||
| fuzzystrmatch | ||
| hstore | ||
| hstore_plperl | ||
| hstore_plpython | ||
| intagg | ||
| intarray | ||
| isn | ||
| jsonb_plperl | ||
| jsonb_plpython | ||
| lo | ||
| ltree | ||
| ltree_plpython | ||
| oid2name | ||
| pageinspect | ||
| passwordcheck | ||
| pg_buffercache | ||
| pg_freespacemap | ||
| pg_prewarm | ||
| pg_standby | ||
| pg_stat_statements | ||
| pg_trgm | ||
| pg_visibility | ||
| pgcrypto | ||
| pgrowlocks | ||
| pgstattuple | ||
| postgres_fdw | ||
| seg | ||
| sepgsql | ||
| spi | ||
| sslinfo | ||
| start-scripts | ||
| tablefunc | ||
| tcn | ||
| test_decoding | ||
| tsm_system_rows | ||
| tsm_system_time | ||
| unaccent | ||
| uuid-ossp | ||
| vacuumlo | ||
| xml2 | ||
| Makefile | ||
| README | ||
| contrib-global.mk | ||
README
The PostgreSQL contrib tree
---------------------------
This subtree contains porting tools, analysis utilities, and plug-in
features that are not part of the core PostgreSQL system, mainly
because they address a limited audience or are too experimental to be
part of the main source tree. This does not preclude their
usefulness.
User documentation for each module appears in the main SGML
documentation.
When building from the source distribution, these modules are not
built automatically, unless you build the "world" target. You can
also build and install them all by running "make all" and "make
install" in this directory; or to build and install just one selected
module, do the same in that module's subdirectory.
Some directories supply new user-defined functions, operators, or
types. To make use of one of these modules, after you have installed
the code you need to register the new SQL objects in the database
system by executing a CREATE EXTENSION command. In a fresh database,
you can simply do
CREATE EXTENSION module_name;
See the PostgreSQL documentation for more information about this
procedure.