postgresql/src/tools
Heikki Linnakangas 8af2565248 Introduce a new smgr bulk loading facility.
The new facility makes it easier to optimize bulk loading, as the
logic for buffering, WAL-logging, and syncing the relation only needs
to be implemented once. It's also less error-prone: We have had a
number of bugs in how a relation is fsync'd - or not - at the end of a
bulk loading operation. By centralizing that logic to one place, we
only need to write it correctly once.

The new facility is faster for small relations: Instead of of calling
smgrimmedsync(), we register the fsync to happen at next checkpoint,
which avoids the fsync latency. That can make a big difference if you
are e.g. restoring a schema-only dump with lots of relations.

It is also slightly more efficient with large relations, as the WAL
logging is performed multiple pages at a time. That avoids some WAL
header overhead. The sorted GiST index build did that already, this
moves the buffering to the new facility.

The changes to pageinspect GiST test needs an explanation: Before this
patch, the sorted GiST index build set the LSN on every page to the
special GistBuildLSN value, not the LSN of the WAL record, even though
they were WAL-logged. There was no particular need for it, it just
happened naturally when we wrote out the pages before WAL-logging
them. Now we WAL-log the pages first, like in B-tree build, so the
pages are stamped with the record's real LSN. When the build is not
WAL-logged, we still use GistBuildLSN. To make the test output
predictable, use an unlogged index.

Reviewed-by: Andres Freund
Discussion: https://www.postgresql.org/message-id/30e8f366-58b3-b239-c521-422122dd5150%40iki.fi
2024-02-23 16:10:51 +02:00
..
ci Remove obsolete script related to MSVC build system 2024-02-11 23:43:50 +01:00
editors Make Emacs perl-mode indent more like perltidy. 2019-01-13 11:32:31 -08:00
ifaddrs Update copyright for 2024 2024-01-03 20:49:05 -05:00
perlcheck Remove MSVC scripts 2023-12-20 09:44:37 +09:00
pg_bsd_indent Update copyright for 2024 2024-01-03 20:49:05 -05:00
pginclude Generate syscache info from catalog files 2024-01-23 07:31:06 +01:00
pgindent Introduce a new smgr bulk loading facility. 2024-02-23 16:10:51 +02:00
PerfectHash.pm Update copyright for 2024 2024-01-03 20:49:05 -05:00
RELEASE_CHANGES Stop generating plain-text INSTALL instructions. 2023-12-22 13:32:15 -05:00
ccsym tools/ccsym: update for modern versions of gcc 2015-01-20 13:02:58 -05:00
check_bison_recursion.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
codelines Fix remaining stray references to CVS. 2010-09-22 19:51:39 -04:00
copyright.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
darwin_sysroot Move darwin sysroot determination into separate file 2022-09-01 16:54:19 -07:00
find_badmacros Remove cvs keywords from all files. 2010-09-20 22:08:53 +02:00
find_meson meson: Add initial version of meson based build system 2022-09-21 22:37:17 -07:00
find_static Fix `trap` in a few shell scripts 2022-09-20 18:50:16 +02:00
find_typedef Refer to OS X as "macOS", except for the port name which is still "darwin". 2016-09-25 15:40:57 -04:00
fix-old-flex-code.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
gen_export.pl Add copyright notices to a few perl scripts that don't have them 2024-01-05 13:15:50 +00:00
gen_keywordlist.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
git-external-diff Preserve information on use of git-external-diff 2018-05-24 23:45:31 +09:30
git_changelog Update copyright for 2024 2024-01-03 20:49:05 -05:00
install_files meson: make install_test_files more generic, rename to install_files 2023-03-23 21:20:18 -07:00
make_ctags Fix make_etags breakage on certain platforms. 2023-06-14 11:02:50 +09:00
make_etags Fix make_etags failure on Mac. 2023-02-15 09:52:42 +09:00
make_mkid Add another pgdefine path check, and a cvs-git change. 2011-08-26 21:52:35 -04:00
mark_pgdllimport.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
msvc_gendef.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
pgflex meson: Add initial version of meson based build system 2022-09-21 22:37:17 -07:00
pgtest pgtest: fix spacing 2023-08-14 14:03:29 -04:00
rcgen meson: Add windows resource files 2022-10-05 09:56:05 -07:00
testint128.c Update copyright for 2024 2024-01-03 20:49:05 -05:00
testwrap Explicitly skip TAP tests under Meson if disabled 2023-11-16 08:14:33 +01:00
valgrind.supp Record dependencies of a cast on other casts that it requires. 2022-10-17 14:02:05 -04:00
version_stamp.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
win32tzlist.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00