postgresql/src/bin/pg_rewind/rewind_source.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

87 lines
2.9 KiB
C
Raw Normal View History

/*-------------------------------------------------------------------------
*
* rewind_source.h
* Abstraction for fetching from source server.
*
* The source server can be either a libpq connection to a live system,
* or a local data directory. The 'rewind_source' struct abstracts the
* operations to fetch data from the source system, so that the rest of
* the code doesn't need to care what kind of a source its dealing with.
*
* Copyright (c) 2013-2023, PostgreSQL Global Development Group
*
*-------------------------------------------------------------------------
*/
#ifndef REWIND_SOURCE_H
#define REWIND_SOURCE_H
#include "access/xlogdefs.h"
#include "file_ops.h"
#include "filemap.h"
#include "libpq-fe.h"
typedef struct rewind_source
{
/*
* Traverse all files in the source data directory, and call 'callback' on
* each file.
*/
void (*traverse_files) (struct rewind_source *,
process_file_callback_t callback);
/*
* Fetch a single file into a malloc'd buffer. The file size is returned
* in *filesize. The returned buffer is always zero-terminated, which is
* handy for text files.
*/
char *(*fetch_file) (struct rewind_source *, const char *path,
size_t *filesize);
/*
* Request to fetch (part of) a file in the source system, specified by an
* offset and length, and write it to the same offset in the corresponding
* target file. The source implementation may queue up the request and
* execute it later when convenient. Call finish_fetch() to flush the
* queue and execute all requests.
*/
void (*queue_fetch_range) (struct rewind_source *, const char *path,
off_t offset, size_t len);
pg_rewind: Fetch small files according to new size. There's a race condition if a file changes in the source system after we have collected the file list. If the file becomes larger, we only fetched up to its original size. That can easily result in a truncated file. That's not a problem for relation files, files in pg_xact, etc. because any actions on them will be replayed from the WAL. However, configuration files are affected. This commit mitigates the race condition by fetching small files in whole, even if they have grown. A test is added in which an extra file copied is concurrently grown with the output of pg_rewind thus guaranteeing it to have changed in size during the operation. This is not a full fix: we still believe the original file size for files larger than 1 MB. That should be enough for configuration files, and doing more than that would require big changes to the chunking logic in libpq_source.c. This mitigates the race condition if the file is modified between the original scan of files and copying the file, but there's still a race condition if a file is changed while it's being copied. That's a much smaller window, though, and pg_basebackup has the same issue. This race can be seen with pg_auto_failover, which frequently uses ALTER SYSTEM, which updates postgresql.auto.conf. Often, pg_rewind will fail, because the postgresql.auto.conf file changed concurrently and a partial version of it was copied to the target. The partial file would fail to parse, preventing the server from starting up. Author: Heikki Linnakangas Reviewed-by: Cary Huang Discussion: https://postgr.es/m/f67feb24-5833-88cb-1020-19a4a2b83ac7%40iki.fi
2022-04-05 14:45:31 +02:00
/*
* Like queue_fetch_range(), but requests replacing the whole local file
* from the source system. 'len' is the expected length of the file,
* although when the source is a live server, the file may change
* concurrently. The implementation is not obliged to copy more than 'len'
* bytes, even if the file is larger. However, to avoid copying a
* truncated version of the file, which can cause trouble if e.g. a
* configuration file is modified concurrently, the implementation should
* try to copy the whole file, even if it's larger than expected.
*/
void (*queue_fetch_file) (struct rewind_source *, const char *path,
size_t len);
/*
* Execute all requests queued up with queue_fetch_range().
*/
void (*finish_fetch) (struct rewind_source *);
/*
* Get the current WAL insert position in the source system.
*/
XLogRecPtr (*get_current_wal_insert_lsn) (struct rewind_source *);
/*
* Free this rewind_source object.
*/
void (*destroy) (struct rewind_source *);
} rewind_source;
/* in libpq_source.c */
extern rewind_source *init_libpq_source(PGconn *conn);
/* in local_source.c */
extern rewind_source *init_local_source(const char *datadir);
#endif /* REWIND_SOURCE_H */