Commit Graph

6041 Commits

Author SHA1 Message Date
greatroar 2e0f1f5113 repository: Remove RunWorkers, report ctx.Err()
This removes RunWorkers, which had become mere overhead by successive
refactors. It also ensures that each former user of that function
returns any context error that occurs, so failure to complete an
operation is always reported as an error.
2022-05-10 22:26:00 +02:00
MichaelEischer 47c56dea5c
Merge pull request #3746 from greatroar/cache-lstat
cache: Don't Lstat before creating CACHEDIR.TAG
2022-05-10 20:31:15 +02:00
Alexander Neumann c270ab1e08
Merge pull request #3744 from MichaelEischer/fix-windows-temp-file
Fix error on temp file deletion on windows
2022-05-10 19:53:25 +02:00
greatroar 2da377c582 cache: Don't Lstat before creating the tag file
The tag file is opened with O_CREATE|O_EXCL and ErrExist is handled, so
we don't need to check for existence first.
2022-05-10 18:52:39 +02:00
Michael Eischer ae7e51382a Fix error on temp file deletion on windows
Apparently it can take a moment between closing a tempfile marked as
DELETE_ON_CLOSE and it actually being deleted. During that time the file
is inaccessible. Thus just skip deleting the temp file on windows.
2022-05-09 22:43:26 +02:00
Michael Eischer 5c6db534d4 Add compression migration support to changelog 2022-05-09 22:39:02 +02:00
Michael Eischer c1bbbcd0dc migrate: Allow migrations to request a check run
This is currently only used by upgrade_repo_v2.
2022-05-09 22:31:30 +02:00
Michael Eischer 59eb132dcd check: Better differentiate between warnings and errors 2022-05-09 22:31:30 +02:00
Michael Eischer 5815f727ee checker: convert error type to use pointer-receivers 2022-05-09 22:31:30 +02:00
Michael Eischer 4faff0debe doc: Describe repository upgrade process 2022-05-09 22:31:30 +02:00
Michael Eischer e36a40db10 upgrade_repo_v2: Use atomic replace for supported backends 2022-05-09 22:31:30 +02:00
Michael Eischer 7559d2f105 Document repository version and minimum restic version 2022-05-09 22:31:30 +02:00
Michael Eischer 381bd94c6c prune: Add option to repack uncompressed data 2022-05-09 22:31:30 +02:00
Michael Eischer 5406743102 prune: Automatically repack uncompressed trees for repo v2
Tree packs are cached locally at clients and thus benefit a lot from
being compressed. Ensure this be having prune always repack pack files
containing uncompressed trees.
2022-05-09 22:31:30 +02:00
Alexander Neumann c8c0d659ec Add migration to compress all data 2022-05-09 22:31:30 +02:00
Alexander Neumann 8c244214bf Add tests for upgrade migration 2022-05-09 22:31:30 +02:00
Alexander Neumann a5f1d318ac Try to make repo upgrade migration more failsafe 2022-05-09 22:31:30 +02:00
Alexander Neumann 82ed5a3a15 Add repo upgrade migration 2022-05-09 22:31:30 +02:00
Alexander Neumann 3af6c180e4 Improve migrate command 2022-05-09 22:31:30 +02:00
Michael Eischer 92816fa966 init: Enable compression support by default 2022-05-09 22:31:30 +02:00
MichaelEischer ab49c14621
Merge pull request #3740 from MichaelEischer/fix-restore-size-stats
Fix restore size calculation for multiple snapshots
2022-05-09 21:39:45 +02:00
Michael Eischer 2c07f7fff3 stats: hardlinks only reduce restore within a snapshot
The `stats` command checks inodes to not count hardlinked files multiple
times into the restore size. This check applies across all snapshots and
not only within snapshots. As a result the result size was far too low
when calculating it for multiple snapshots and it would vary depending
on the order in which snapshots were listed.
2022-05-09 21:26:24 +02:00
Lorenz Bausch 9fb81c4246
Validate exclude patterns 2022-05-07 21:12:47 +02:00
Lorenz Bausch e7fd200237
Keep original pattern for later use 2022-05-07 21:08:09 +02:00
MichaelEischer cc8a03b1d0
Merge pull request #3735 from HenrikBengtsson/master
DOCS: Incorrectly used SI units when IEC units were meant (fix #3669)
2022-05-07 11:15:27 +02:00
Henrik Bengtsson 9bb532672a DOCS: Incorrectly used SI units when IEC units were meant (fix #3669) 2022-05-02 21:21:39 -07:00
MichaelEischer 26c333325c
Merge pull request #3715 from ema/master
doc: specify AWS Region via AWS_DEFAULT_REGION
2022-05-02 21:46:56 +02:00
Emanuele Rocca 38c0531b52 doc: specify AWS Region via AWS_DEFAULT_REGION
If no specific Region is mentioned in RESTIC_REPOSITORY, AWS defaults to
us-east-1. For this reason, users that follow the tutorial and create
their S3 bucket in any other region get the following error:

"Fatal: create repository at [...] client.BucketExists"

Explicitly specifying the AWS region name fixes the issue.
2022-05-02 21:26:58 +02:00
Alexander Neumann fb5b9370f3
Merge pull request #3728 from MichaelEischer/debug-examine-upload
Add support to reupload blobs in `debug examine`
2022-04-30 20:39:19 +02:00
Alexander Neumann ffbd48c0c6
Merge pull request #3481 from MichaelEischer/recover-enospace
Recover from no free space errors
2022-04-30 20:22:26 +02:00
Michael Eischer 95bcc9ea31 debug: Support pack ID prefixes in `debug examine` 2022-04-30 20:20:31 +02:00
Michael Eischer 2d6a943911 debug: Add switch to upload blobs extracted by debug examine
This simplifies salvaging a damaged pack file. Reuploading tree blobs
was previously not possible.
2022-04-30 20:20:31 +02:00
Alexander Neumann 9af499d8a4 Tidy go.sum 2022-04-30 20:16:09 +02:00
Alexander Neumann 2e3d23c1d7 Update github.com/klauspost/compress 2022-04-30 20:03:21 +02:00
Michael Eischer dbbeac7174 prune: Add unsafe option to recover from no free space
The new option allows prune to operate with nearly no scratch space by only removing
no longer necessary pack files and first deleting the index before
rebuilding it. By first deleting the index it becomes safe to just
delete no longer necessary pack files. However, as a downside there's
now the risk that the repository becomes inaccessible if prune fails.

To recover from that problem a user might have to manually delete the
repository index and then run (a full) `rebuild-index` again.
2022-04-30 19:21:07 +02:00
Michael Eischer cf5cb673fb repository: Use existing method to collect pack ids 2022-04-30 19:14:21 +02:00
Michael Eischer b335cb6285 repository: Refactor index IDs collection 2022-04-30 19:14:21 +02:00
MichaelEischer 9c047f170a
Merge pull request #3419 from DanielG/relax-file-modes
Enable admin to decide group access to repository files
2022-04-30 16:39:37 +02:00
Daniel Gröber f31b4f29c1 Use config file modes to derive new dir/file modes
Fixes #2351
2022-04-30 15:59:51 +02:00
Alexander Neumann 71c653f9e0
Merge pull request #3727 from MichaelEischer/changelog-3475
Add changelog for local/sftp connection limit
2022-04-30 14:05:21 +02:00
Michael Eischer 29a8f92967 Add changelog for local/sftp connection limit 2022-04-30 13:36:47 +02:00
MichaelEischer ac9324aeaf
Merge pull request #3666 from MichaelEischer/compression
Implement compression support
2022-04-30 11:49:05 +02:00
Alexander Neumann dc5adef255 Add documentation for --repository-version 2022-04-30 11:34:10 +02:00
Michael Eischer 4b01b06f2f repository: Test compressed blobs in StreamPack 2022-04-30 11:34:10 +02:00
Michael Eischer bcab548617 pack: slightly expand testing of compressed blobs 2022-04-30 11:34:10 +02:00
Michael Eischer ec2b25565a repository: test uncompressedLength field and index example 2022-04-30 11:34:10 +02:00
Michael Eischer 9ffb8920f1 repository: run blackbox tests using old and new repo version 2022-04-30 11:34:10 +02:00
Michael Eischer abe5935693 repository: unify repository version-specific initialization
Mark the master index as compressed also when initializing a new
repository. This is only relevant for testing.
2022-04-30 11:34:10 +02:00
Alexander Neumann 8776031f96 Leave allocating slices to the decompress code 2022-04-30 11:34:10 +02:00
Alexander Neumann 5eb05a0afe Configure zstd encoder/decoder 2022-04-30 11:34:10 +02:00