Go to file
Michael Paquier 71e4cc6b8e Optimize WAL insertion lock acquisition and release with some atomics
The WAL insertion lock variable insertingAt is currently being read
and written with the help of the LWLock wait list lock to avoid any read
of torn values.  This wait list lock can become a point of contention on
a highly concurrent write workloads.

This commit switches insertingAt to a 64b atomic variable that provides
torn-free reads/writes.  On platforms without 64b atomic support, the
fallback implementation uses spinlocks to provide the same guarantees
for the values read.  LWLockWaitForVar(), through
LWLockConflictsWithVar(), reads the new value to check if it still needs
to wait with a u64 atomic operation.  LWLockUpdateVar() updates the
variable before waking up the waiters with an exchange_u64 (full memory
barrier).  LWLockReleaseClearVar() now uses also an exchange_u64 to
reset the variable.  Before this commit, all these steps relied on
LWLockWaitListLock() and LWLockWaitListUnlock().

This reduces contention on LWLock wait list lock and improves
performance of highly-concurrent write workloads.  Here are some
numbers using pg_logical_emit_message() (HEAD at d6677b93) with various
arbitrary record lengths and clients up to 1k on a rather-large machine
(64 vCPUs, 512GB of RAM, 16 cores per sockets, 2 sockets), in terms of
TPS numbers coming from pgbench:
 message_size_b     |     16 |     64 |    256 |   1024
--------------------+--------+--------+--------+-------
 patch_4_clients    |  83830 |  82929 |  80478 |  73131
 patch_16_clients   | 267655 | 264973 | 250566 | 213985
 patch_64_clients   | 380423 | 378318 | 356907 | 294248
 patch_256_clients  | 360915 | 354436 | 326209 | 263664
 patch_512_clients  | 332654 | 321199 | 287521 | 240128
 patch_1024_clients | 288263 | 276614 | 258220 | 217063
 patch_2048_clients | 252280 | 243558 | 230062 | 192429
 patch_4096_clients | 212566 | 213654 | 205951 | 166955
 head_4_clients     |  83686 |  83766 |  81233 |  73749
 head_16_clients    | 266503 | 265546 | 249261 | 213645
 head_64_clients    | 366122 | 363462 | 341078 | 261707
 head_256_clients   | 132600 | 132573 | 134392 | 165799
 head_512_clients   | 118937 | 114332 | 116860 | 150672
 head_1024_clients  | 133546 | 115256 | 125236 | 151390
 head_2048_clients  | 137877 | 117802 | 120909 | 138165
 head_4096_clients  | 113440 | 115611 | 120635 | 114361

Bharath has been measuring similar improvements, where the limit of the
WAL insertion lock begins to be felt when more than 256 concurrent
clients are involved in this specific workload.

An extra patch has been discussed to introduce a fast-exit path in
LWLockUpdateVar() when there are no waiters, still this does not
influence the write-heavy workload cases discussed as there are always
waiters.  This will be considered separately.

Author: Bharath Rupireddy
Reviewed-by: Nathan Bossart, Andres Freund, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACVF+6jLvqKe6xhDzCCkr=rfd6upaGc3477Pji1Ke9G7Bg@mail.gmail.com
2023-07-25 13:38:58 +09:00
config All supported systems have locale_t. 2023-07-09 11:55:18 +12:00
contrib Make test_decoding ddl.out shorter 2023-07-24 17:48:06 +02:00
doc pgbench: Use COPY for client-side data generation 2023-07-24 13:48:22 +09:00
src Optimize WAL insertion lock acquisition and release with some atomics 2023-07-25 13:38:58 +09:00
.cirrus.yml cirrus/freebsd: define ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS 2023-07-03 10:35:48 +02:00
.dir-locals.el Make Emacs perl-mode indent more like perltidy. 2019-01-13 11:32:31 -08:00
.editorconfig Add .editorconfig 2019-12-18 09:13:13 +01:00
.git-blame-ignore-revs Add b334612b8 to .git-blame-ignore-revs. 2023-06-20 09:52:52 -04:00
.gitattributes gitattributes: Ignore imported pg_bsd_indent code for whitespace checks 2023-02-22 09:00:28 +01:00
.gitignore Update top-level .gitignore. 2022-12-04 15:23:00 -05:00
aclocal.m4 autoconf: Move export_dynamic determination to configure 2022-12-06 18:55:28 -08:00
configure Remove --disable-thread-safety and related code. 2023-07-12 08:20:43 +12:00
configure.ac Remove --disable-thread-safety and related code. 2023-07-12 08:20:43 +12:00
COPYRIGHT Update copyright for 2023 2023-01-02 15:00:37 -05:00
GNUmakefile.in Integrate pg_bsd_indent into our build/test infrastructure. 2023-02-12 12:22:21 -05:00
HISTORY Canonicalize some URLs 2020-02-10 20:47:50 +01:00
Makefile Dynamically find correct installation docs in Makefile. 2022-01-19 14:48:25 +01:00
meson_options.txt meson: Attach colon to keyword argument 2023-06-29 12:53:41 +02:00
meson.build meson: Tie adding C++ support to the llvm Meson option 2023-07-12 16:26:02 -07:00
README Canonicalize some URLs 2020-02-10 20:47:50 +01:00
README.git Canonicalize some URLs 2020-02-10 20:47:50 +01:00

PostgreSQL Database Management System
=====================================

This directory contains the source code distribution of the PostgreSQL
database management system.

PostgreSQL is an advanced object-relational database management system
that supports an extended subset of the SQL standard, including
transactions, foreign keys, subqueries, triggers, user-defined types
and functions.  This distribution also contains C language bindings.

PostgreSQL has many language interfaces, many of which are listed here:

	https://www.postgresql.org/download/

See the file INSTALL for instructions on how to build and install
PostgreSQL.  That file also lists supported operating systems and
hardware platforms and contains information regarding any other
software packages that are required to build or run the PostgreSQL
system.  Copyright and license information can be found in the
file COPYRIGHT.  A comprehensive documentation set is included in this
distribution; it can be read as described in the installation
instructions.

The latest version of this software may be obtained at
https://www.postgresql.org/download/.  For more information look at our
web site located at https://www.postgresql.org/.