Go to file
Masahiko Sawada 5bec1d6bc5 Improve eviction algorithm in ReorderBuffer using max-heap for many subtransactions.
Previously, when selecting the transaction to evict during logical
decoding, we check all transactions to find the largest
transaction. This could lead to a significant replication lag
especially in the case where there are many subtransactions.

This commit improves the eviction algorithm in ReorderBuffer using the
max-heap with transaction size as the key to efficiently find the
largest transaction.

The max-heap starts with empty. While the max-heap is empty, we don't
do anything for the max-heap when updating the memory
counter. Therefore, we get the largest transaction in O(N) time, where
N is the number of transactions including top-level transactions and
subtransactions.

We build the max-heap just before selecting the largest transactions
if the number of transactions being decoded is higher than the
threshold, MAX_HEAP_TXN_COUNT_THRESHOLD. After building the max-heap,
we also update the max-heap when updating the memory counter. The
intention is to efficiently find the largest transaction in O(1) time
instead of incurring the cost of memory counter updates (O(log
N)). Once the number of transactions got lower than the threshold, we
reset the max-heap.

The performance benchmark results showed significant speed up (more
than x30 speed up on my machine) in decoding a transaction with 100k
subtransactions, whereas there is no visible overhead in other cases.

Reviewed-by: Amit Kapila, Hayato Kuroda, Vignesh C, Ajin Cherian,
Tomas Vondra, Shubham Khanna, Peter Smith, Álvaro Herrera,
Euler Taveira
Discussion: https://postgr.es/m/CAD21AoAfKTgrBrLq96GcTv9d6k97zaQcDM-rxfKEt4GSe0qnaQ%40mail.gmail.com
2024-04-03 11:40:42 +09:00
config Remove AIX support 2024-02-28 15:17:23 +04:00
contrib Use streaming I/O in pg_prewarm. 2024-04-03 00:57:34 +13:00
doc Implement pg_wal_replay_wait() stored procedure 2024-04-02 22:48:03 +03:00
src Improve eviction algorithm in ReorderBuffer using max-heap for many subtransactions. 2024-04-03 11:40:42 +09:00
.cirrus.star Remove duplicate words in docs and code comments. 2023-10-09 09:18:47 +05:30
.cirrus.tasks.yml ci: macos: Choose python version 2024-03-25 16:39:08 -07:00
.cirrus.yml ci: Make compute resources for CI configurable 2023-08-23 15:15:28 -07:00
.dir-locals.el Make Emacs perl-mode indent more like perltidy. 2019-01-13 11:32:31 -08:00
.editorconfig Add .editorconfig 2019-12-18 09:13:13 +01:00
.git-blame-ignore-revs Add commit 64e401b62b to .git-blame-ignore-revs. 2024-03-26 15:08:47 -05:00
.gitattributes Remove non-existing file from .gitattributes 2024-02-16 11:39:09 +01:00
.gitignore Update top-level .gitignore. 2022-12-04 15:23:00 -05:00
COPYRIGHT Update copyright for 2024 2024-01-03 20:49:05 -05:00
GNUmakefile.in make dist uses git archive 2024-03-25 06:29:53 +01:00
HISTORY Canonicalize some URLs 2020-02-10 20:47:50 +01:00
Makefile Remove AIX support 2024-02-28 15:17:23 +04:00
README.md Revise the style of a paragraph in README.md. 2024-03-21 10:16:41 -05:00
aclocal.m4 autoconf: Move export_dynamic determination to configure 2022-12-06 18:55:28 -08:00
configure Add --copy-file-range option to pg_upgrade. 2024-03-06 12:01:01 +13:00
configure.ac Add --copy-file-range option to pg_upgrade. 2024-03-06 12:01:01 +13:00
meson.build make dist uses git archive 2024-03-25 06:29:53 +01:00
meson_options.txt Add backend support for injection points 2024-01-22 10:15:50 +09:00

README.md

PostgreSQL Database Management System

This directory contains the source code distribution of the PostgreSQL database management system.

PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions. This distribution also contains C language bindings.

Copyright and license information can be found in the file COPYRIGHT.

General documentation about this version of PostgreSQL can be found at https://www.postgresql.org/docs/devel/. In particular, information about building PostgreSQL from the source code can be found at https://www.postgresql.org/docs/devel/installation.html.

The latest version of this software, and related software, may be obtained at https://www.postgresql.org/download/. For more information look at our web site located at https://www.postgresql.org/.