2023-12-20 15:49:12 +01:00
|
|
|
/*
|
|
|
|
* Copy entire files.
|
|
|
|
*
|
2024-01-04 02:49:05 +01:00
|
|
|
* Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
|
2023-12-20 15:49:12 +01:00
|
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
|
|
|
*
|
|
|
|
* src/bin/pg_combinebackup/copy_file.h
|
|
|
|
*
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
*/
|
|
|
|
#include "postgres_fe.h"
|
|
|
|
|
|
|
|
#ifdef HAVE_COPYFILE_H
|
|
|
|
#include <copyfile.h>
|
|
|
|
#endif
|
|
|
|
#include <fcntl.h>
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
#include <limits.h>
|
2023-12-20 15:49:12 +01:00
|
|
|
#include <sys/stat.h>
|
|
|
|
#include <unistd.h>
|
|
|
|
|
|
|
|
#include "common/file_perm.h"
|
|
|
|
#include "common/logging.h"
|
|
|
|
#include "copy_file.h"
|
|
|
|
|
|
|
|
static void copy_file_blocks(const char *src, const char *dst,
|
|
|
|
pg_checksum_context *checksum_ctx);
|
|
|
|
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
static void copy_file_clone(const char *src, const char *dst,
|
|
|
|
pg_checksum_context *checksum_ctx);
|
|
|
|
|
|
|
|
static void copy_file_by_range(const char *src, const char *dst,
|
|
|
|
pg_checksum_context *checksum_ctx);
|
|
|
|
|
2023-12-20 15:49:12 +01:00
|
|
|
#ifdef WIN32
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
static void copy_file_copyfile(const char *src, const char *dst,
|
|
|
|
pg_checksum_context *checksum_ctx);
|
2023-12-20 15:49:12 +01:00
|
|
|
#endif
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Copy a regular file, optionally computing a checksum, and emitting
|
|
|
|
* appropriate debug messages. But if we're in dry-run mode, then just emit
|
|
|
|
* the messages and don't copy anything.
|
|
|
|
*/
|
|
|
|
void
|
|
|
|
copy_file(const char *src, const char *dst,
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
pg_checksum_context *checksum_ctx,
|
|
|
|
CopyMethod copy_method, bool dry_run)
|
2023-12-20 15:49:12 +01:00
|
|
|
{
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
char *strategy_name = NULL;
|
|
|
|
void (*strategy_implementation) (const char *, const char *,
|
|
|
|
pg_checksum_context *checksum_ctx) = NULL;
|
|
|
|
|
2023-12-20 15:49:12 +01:00
|
|
|
/*
|
|
|
|
* In dry-run mode, we don't actually copy anything, nor do we read any
|
|
|
|
* data from the source file, but we do verify that we can open it.
|
|
|
|
*/
|
|
|
|
if (dry_run)
|
|
|
|
{
|
|
|
|
int fd;
|
|
|
|
|
|
|
|
if ((fd = open(src, O_RDONLY | PG_BINARY, 0)) < 0)
|
|
|
|
pg_fatal("could not open \"%s\": %m", src);
|
|
|
|
if (close(fd) < 0)
|
|
|
|
pg_fatal("could not close \"%s\": %m", src);
|
|
|
|
}
|
|
|
|
|
|
|
|
#ifdef WIN32
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
copy_method = COPY_METHOD_COPYFILE;
|
2023-12-20 15:49:12 +01:00
|
|
|
#endif
|
|
|
|
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
/* Determine the name of the copy strategy for use in log messages. */
|
|
|
|
switch (copy_method)
|
|
|
|
{
|
|
|
|
case COPY_METHOD_CLONE:
|
|
|
|
strategy_name = "clone";
|
|
|
|
strategy_implementation = copy_file_clone;
|
|
|
|
break;
|
|
|
|
case COPY_METHOD_COPY:
|
|
|
|
/* leave NULL for simple block-by-block copy */
|
|
|
|
strategy_implementation = copy_file_blocks;
|
|
|
|
break;
|
|
|
|
case COPY_METHOD_COPY_FILE_RANGE:
|
|
|
|
strategy_name = "copy_file_range";
|
|
|
|
strategy_implementation = copy_file_by_range;
|
|
|
|
break;
|
|
|
|
#ifdef WIN32
|
|
|
|
case COPY_METHOD_COPYFILE:
|
|
|
|
strategy_name = "CopyFile";
|
|
|
|
strategy_implementation = copy_file_copyfile;
|
|
|
|
break;
|
|
|
|
#endif
|
2023-12-20 15:49:12 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
if (dry_run)
|
|
|
|
{
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
if (strategy_name)
|
|
|
|
pg_log_debug("would copy \"%s\" to \"%s\" using strategy %s",
|
|
|
|
src, dst, strategy_name);
|
|
|
|
else
|
2023-12-20 15:49:12 +01:00
|
|
|
pg_log_debug("would copy \"%s\" to \"%s\"",
|
|
|
|
src, dst);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
if (strategy_name)
|
|
|
|
pg_log_debug("copying \"%s\" to \"%s\" using strategy %s",
|
|
|
|
src, dst, strategy_name);
|
|
|
|
else if (checksum_ctx->type == CHECKSUM_TYPE_NONE)
|
2023-12-20 15:49:12 +01:00
|
|
|
pg_log_debug("copying \"%s\" to \"%s\"",
|
|
|
|
src, dst);
|
|
|
|
else
|
|
|
|
pg_log_debug("copying \"%s\" to \"%s\" and checksumming with %s",
|
|
|
|
src, dst, pg_checksum_type_name(checksum_ctx->type));
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
|
|
|
|
strategy_implementation(src, dst, checksum_ctx);
|
2023-12-20 15:49:12 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
/*
|
|
|
|
* Calculate checksum for the src file.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
checksum_file(const char *src, pg_checksum_context *checksum_ctx)
|
|
|
|
{
|
|
|
|
int src_fd;
|
|
|
|
uint8 *buffer;
|
|
|
|
const int buffer_size = 50 * BLCKSZ;
|
|
|
|
ssize_t rb;
|
|
|
|
|
|
|
|
/* bail out if no checksum needed */
|
|
|
|
if (checksum_ctx->type == CHECKSUM_TYPE_NONE)
|
|
|
|
return;
|
|
|
|
|
|
|
|
if ((src_fd = open(src, O_RDONLY | PG_BINARY, 0)) < 0)
|
|
|
|
pg_fatal("could not open file \"%s\": %m", src);
|
|
|
|
|
|
|
|
buffer = pg_malloc(buffer_size);
|
|
|
|
|
|
|
|
while ((rb = read(src_fd, buffer, buffer_size)) > 0)
|
|
|
|
{
|
|
|
|
if (pg_checksum_update(checksum_ctx, buffer, rb) < 0)
|
|
|
|
pg_fatal("could not update checksum of file \"%s\"", src);
|
|
|
|
}
|
|
|
|
|
|
|
|
if (rb < 0)
|
|
|
|
pg_fatal("could not read file \"%s\": %m", src);
|
|
|
|
|
|
|
|
pg_free(buffer);
|
|
|
|
close(src_fd);
|
|
|
|
}
|
|
|
|
|
2023-12-20 15:49:12 +01:00
|
|
|
/*
|
|
|
|
* Copy a file block by block, and optionally compute a checksum as we go.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
copy_file_blocks(const char *src, const char *dst,
|
|
|
|
pg_checksum_context *checksum_ctx)
|
|
|
|
{
|
|
|
|
int src_fd;
|
|
|
|
int dest_fd;
|
|
|
|
uint8 *buffer;
|
|
|
|
const int buffer_size = 50 * BLCKSZ;
|
|
|
|
ssize_t rb;
|
|
|
|
unsigned offset = 0;
|
|
|
|
|
|
|
|
if ((src_fd = open(src, O_RDONLY | PG_BINARY, 0)) < 0)
|
|
|
|
pg_fatal("could not open file \"%s\": %m", src);
|
|
|
|
|
|
|
|
if ((dest_fd = open(dst, O_WRONLY | O_CREAT | O_EXCL | PG_BINARY,
|
|
|
|
pg_file_create_mode)) < 0)
|
|
|
|
pg_fatal("could not open file \"%s\": %m", dst);
|
|
|
|
|
|
|
|
buffer = pg_malloc(buffer_size);
|
|
|
|
|
|
|
|
while ((rb = read(src_fd, buffer, buffer_size)) > 0)
|
|
|
|
{
|
|
|
|
ssize_t wb;
|
|
|
|
|
|
|
|
if ((wb = write(dest_fd, buffer, rb)) != rb)
|
|
|
|
{
|
|
|
|
if (wb < 0)
|
|
|
|
pg_fatal("could not write file \"%s\": %m", dst);
|
|
|
|
else
|
|
|
|
pg_fatal("could not write file \"%s\": wrote only %d of %d bytes at offset %u",
|
|
|
|
dst, (int) wb, (int) rb, offset);
|
|
|
|
}
|
|
|
|
|
|
|
|
if (pg_checksum_update(checksum_ctx, buffer, rb) < 0)
|
|
|
|
pg_fatal("could not update checksum of file \"%s\"", dst);
|
|
|
|
|
|
|
|
offset += rb;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (rb < 0)
|
|
|
|
pg_fatal("could not read file \"%s\": %m", dst);
|
|
|
|
|
|
|
|
pg_free(buffer);
|
|
|
|
close(src_fd);
|
|
|
|
close(dest_fd);
|
|
|
|
}
|
|
|
|
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
/*
|
|
|
|
* copy_file_clone
|
|
|
|
* Clones/reflinks a file from src to dest.
|
|
|
|
*
|
|
|
|
* If needed, also reads the file and calculates the checksum.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
copy_file_clone(const char *src, const char *dest,
|
|
|
|
pg_checksum_context *checksum_ctx)
|
|
|
|
{
|
|
|
|
#if defined(HAVE_COPYFILE) && defined(COPYFILE_CLONE_FORCE)
|
|
|
|
if (copyfile(src, dest, NULL, COPYFILE_CLONE_FORCE) < 0)
|
|
|
|
pg_fatal("error while cloning file \"%s\" to \"%s\": %m", src, dest);
|
|
|
|
#elif defined(__linux__) && defined(FICLONE)
|
|
|
|
{
|
|
|
|
if ((src_fd = open(src, O_RDONLY | PG_BINARY, 0)) < 0)
|
|
|
|
pg_fatal("could not open file \"%s\": %m", src);
|
|
|
|
|
|
|
|
if ((dest_fd = open(dest, O_RDWR | O_CREAT | O_EXCL | PG_BINARY,
|
|
|
|
pg_file_create_mode)) < 0)
|
|
|
|
pg_fatal("could not create file \"%s\": %m", dest);
|
|
|
|
|
|
|
|
if (ioctl(dest_fd, FICLONE, src_fd) < 0)
|
|
|
|
{
|
|
|
|
int save_errno = errno;
|
|
|
|
|
|
|
|
unlink(dest);
|
|
|
|
|
|
|
|
pg_fatal("error while cloning file \"%s\" to \"%s\": %s",
|
|
|
|
src, dest);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
#else
|
|
|
|
pg_fatal("file cloning not supported on this platform");
|
|
|
|
#endif
|
|
|
|
|
|
|
|
/* if needed, calculate checksum of the file */
|
|
|
|
checksum_file(src, checksum_ctx);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* copy_file_by_range
|
|
|
|
* Copies a file from src to dest using copy_file_range system call.
|
|
|
|
*
|
|
|
|
* If needed, also reads the file and calculates the checksum.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
copy_file_by_range(const char *src, const char *dest,
|
|
|
|
pg_checksum_context *checksum_ctx)
|
|
|
|
{
|
|
|
|
#if defined(HAVE_COPY_FILE_RANGE)
|
|
|
|
int src_fd;
|
|
|
|
int dest_fd;
|
|
|
|
ssize_t nbytes;
|
|
|
|
|
|
|
|
if ((src_fd = open(src, O_RDONLY | PG_BINARY, 0)) < 0)
|
|
|
|
pg_fatal("could not open file \"%s\": %m", src);
|
|
|
|
|
|
|
|
if ((dest_fd = open(dest, O_RDWR | O_CREAT | O_EXCL | PG_BINARY,
|
|
|
|
pg_file_create_mode)) < 0)
|
|
|
|
pg_fatal("could not create file \"%s\": %m", dest);
|
|
|
|
|
|
|
|
do
|
|
|
|
{
|
|
|
|
nbytes = copy_file_range(src_fd, NULL, dest_fd, NULL, SSIZE_MAX, 0);
|
|
|
|
if (nbytes < 0)
|
|
|
|
pg_fatal("error while copying file range from \"%s\" to \"%s\": %m",
|
|
|
|
src, dest);
|
|
|
|
} while (nbytes > 0);
|
|
|
|
|
|
|
|
close(src_fd);
|
|
|
|
close(dest_fd);
|
|
|
|
#else
|
|
|
|
pg_fatal("copy_file_range not supported on this platform");
|
|
|
|
#endif
|
|
|
|
|
|
|
|
/* if needed, calculate checksum of the file */
|
|
|
|
checksum_file(src, checksum_ctx);
|
|
|
|
}
|
|
|
|
|
2023-12-20 15:49:12 +01:00
|
|
|
#ifdef WIN32
|
|
|
|
static void
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
copy_file_copyfile(const char *src, const char *dst,
|
|
|
|
pg_checksum_context *checksum_ctx)
|
2023-12-20 15:49:12 +01:00
|
|
|
{
|
|
|
|
if (CopyFile(src, dst, true) == 0)
|
|
|
|
{
|
|
|
|
_dosmaperr(GetLastError());
|
|
|
|
pg_fatal("could not copy \"%s\" to \"%s\": %m", src, dst);
|
|
|
|
}
|
Allow copying files using clone/copy_file_range
Adds --clone/--copy-file-range options to pg_combinebackup, to allow
copying files using file cloning or copy_file_range(). These methods may
be faster than the standard block-by-block copy, but the main advantage
is that they enable various features provided by CoW filesystems.
This commit only uses these copy methods for files that did not change
and can be copied as a whole from a single backup.
These new copy methods may not be available on all platforms, in which
case the command throws an error (immediately, even if no files would be
copied as a whole). This early failure seems better than failing later
when trying to copy the first file, after performing a lot of work on
earlier files.
If the requested copy method is available, but a checksum needs to be
recalculated (e.g. because of a different checksum type), the file is
still copied using the requested method, but it is also read for the
checksum calculation. Depending on the filesystem this may be more
expensive than just performing the simple copy, but it does enable the
CoW benefits.
Initial patch by Jakub Wartak, various reworks and improvements by me.
Author: Tomas Vondra, Jakub Wartak
Reviewed-by: Thomas Munro, Jakub Wartak, Robert Haas
Discussion: https://postgr.es/m/3024283a-7491-4240-80d0-421575f6bb23%40enterprisedb.com
2024-04-05 18:01:26 +02:00
|
|
|
|
|
|
|
/* if needed, calculate checksum of the file */
|
|
|
|
checksum_file(src, checksum_ctx);
|
2023-12-20 15:49:12 +01:00
|
|
|
}
|
|
|
|
#endif /* WIN32 */
|