Commit Graph

7521 Commits

Author SHA1 Message Date
adrian5 609f84e095 docs: fix formatting 2024-01-22 21:12:12 +01:00
Michael Eischer 6696195f38
Merge pull request #4584 from elkemper/fix-stop-archiving-metadata
S3: Don't archive metadata files on S3 Glacier
2024-01-20 10:30:37 +00:00
Michael Eischer a763a5c67d s3: minor cleanups for archive storage class handling 2024-01-20 11:25:28 +01:00
Vladislav Belous 8ca58b487c S3: do not set storage class for metadata when using archive storage 2024-01-20 11:04:15 +01:00
Michael Eischer 62111f4379
Merge pull request #4625 from MichaelEischer/refactor-streampacks
Refactor repository.StreamPacks
2024-01-19 21:48:37 +01:00
Michael Eischer 2c310a526e repository: Replace StreamPack function with LoadBlobsFromPack method
LoadBlobsFromPack is now part of the repository struct. This ensures
that users of that method don't have to deal will internals of the
repository implementation.

The filerestorer tests now also contain far fewer pack file
implementation details.
2024-01-19 21:40:43 +01:00
Michael Eischer 6b7b5c89e9 repository: prepare StreamPack refactor 2024-01-19 21:40:43 +01:00
Michael Eischer 22d0c3f8dc check: Use PackBlobIterator instead of StreamPack
To only stream the content of a pack file once, check used StreamPack
with a custom pack load function. This combination was always brittle
and complicates using StreamPack everywhere else. Now that StreamPack
internally uses PackBlobIterator use that primitive instead, which is a
much better fit for what the check command requires.
2024-01-19 21:40:36 +01:00
Michael Eischer fb422497af repository: split StreamPack implementation
Move the actual decoding of the pack data into a separate iterator.
2024-01-19 21:39:55 +01:00
Michael Eischer 54c5c72e5a
Merge pull request #4616 from MichaelEischer/fix-rest-connection-close
rest: fix and cleanup closing of http response body
2024-01-19 21:31:35 +01:00
Michael Eischer 5f49eec655
Merge pull request #4615 from MichaelEischer/fix-find-empty-dirs
walker: Remove ignoreTrees functionality
2024-01-19 21:25:40 +01:00
Michael Eischer ec13105093
Merge pull request #4623 from MichaelEischer/docs-verify-release-binaries
Add documentation for the verify-release-binaries.sh script
2024-01-19 21:17:38 +01:00
Michael Eischer bd883caae1 CI: enable bodyclose linter 2024-01-19 21:17:18 +01:00
Michael Eischer b1a8fd1d03 rest: fix and cleanup closing of http response body
If client.Do returns an error, then there's no body that has to be
closed. For requests for which we are not interested in the response
body, immediately drain and close the body to make sure it isn't
forgotten later on.

This change in particular adds the missing `Close()` call for the
`List()` command.
2024-01-19 21:17:17 +01:00
Michael Eischer fdcbb53017 walker: test skipping for root node 2024-01-19 21:16:06 +01:00
Michael Eischer 0b39940fdb walker: Remove ignoreTrees functionality
It was only used in two places:
- stats: apparently as a minor performance optimization, which is
  unlikely to be important
- find: filtered directories would be ignored. However, this
  optimization missed that it is possible that two directories have the
  exact same content. Such directories would be incorrectly ignored too.
  Example:
```
mkdir test test/a test/b
restic backup test
restic find latest test/b
-> incorrectly does not return anything
```

Thus, remove the functionality as it's apparently too complex to use
correctly.
2024-01-19 21:16:06 +01:00
Michael Eischer 147b0e54cb
Merge pull request #4639 from northben/patch-1
Update Backblaze documentation
2024-01-19 20:08:48 +00:00
Ben Northway 5413877d33 Update Backblaze documentation
clarify documentation regarding B2 bucket lifecycle settings. The default lifecycle setting is probably fine for most users now; a custom policy is not necessary.
2024-01-18 16:41:01 -06:00
Michael Eischer 03e06d0797 Merge branch 'patch-release' 2024-01-14 21:38:17 +01:00
Alexander Neumann 0ec9383ba2 Set development version for 0.16.3 2024-01-14 20:21:45 +01:00
Alexander Neumann abca112404 Add version for 0.16.3 2024-01-14 20:21:45 +01:00
Alexander Neumann b70b94507a Generate CHANGELOG.md for 0.16.3 2024-01-14 20:21:19 +01:00
Alexander Neumann d987582594 Prepare changelog for 0.16.3 2024-01-14 20:21:19 +01:00
Leo R. Lundgren ef2e473b99 doc: Polish changelogs 2024-01-10 00:19:07 +01:00
Michael Eischer 7b2de84763
Merge pull request #4618 from MichaelEischer/workaround-rclone-list-errors
rclone: Workaround for incorrect "not found" errors while listing files
2024-01-09 18:28:31 +01:00
Michael Eischer e4bbde7036 rclone: Workaround for incorrect "not found" errors while listing files
rclone returns a "not found" error if an internal error occurs while
listing a folder. Ignoring this error lets restic erroneously think that
there are no files, which can cause `prune` to wipe the whole
repository.
2024-01-09 18:28:17 +01:00
Michael Eischer ec0fb46f6c add changelog for reliable restores 2024-01-09 18:27:48 +01:00
Michael Eischer 103beb96bc restore: separately restore blobs that are frequently referenced
Writing these blobs to their files can take a long time and consequently
cause the backend connection to time out. Avoid that by retrieving these
blobs separately.
2024-01-09 18:27:48 +01:00
Michael Eischer f0f89d7f27 restore: split error reporting from downloadPack 2024-01-09 18:27:48 +01:00
Michael Eischer cf352ccafb restore: cleanup downloadPack 2024-01-09 18:27:48 +01:00
Michael Eischer b856e9489a restore: split downloadPack into smaller methods 2024-01-09 18:27:48 +01:00
Michael Eischer c31e9418ba
Merge pull request #4626 from MichaelEischer/reliable-large-restores
Improve reliability of large restores
2024-01-09 18:23:09 +01:00
Michael Eischer 2e8de9edfd rclone: Workaround for incorrect "not found" errors while listing files
rclone returns a "not found" error if an internal error occurs while
listing a folder. Ignoring this error lets restic erroneously think that
there are no files, which can cause `prune` to wipe the whole
repository.
2024-01-09 18:20:16 +01:00
Michael Eischer ce7db90e08 sync CI and go dependencies with master branch 2024-01-08 21:33:05 +01:00
Michael Eischer 620518aec6 add changelog for better restore error reporting 2024-01-08 21:33:05 +01:00
Michael Eischer f2fafbffaa restore: only report errors for blobs that actually failed to load
Previously, errors would be reported for all blobs of a packfile that
failed to stream. Now, only the not yet processed blobs are reported.
2024-01-08 21:33:05 +01:00
Michael Eischer 7a3a884874 repository: test that StreamPack only delivers blobs once 2024-01-08 21:33:05 +01:00
Michael Eischer 772a907533 repository: StreamPack delivers blobs at most once
If an error occurred while streaming a pack file, this could result in
passing some of the blobs multiple times to the callback function. This
significantly complicates using StreamPack correctly and is unnecessary.
Retries do not change the content of a blob and thus only deliver the
same result over and over again.
2024-01-08 21:33:05 +01:00
Michael Eischer a9446c1184 add changelog for irregular files on windows 2024-01-08 21:33:05 +01:00
Michael Eischer 1bab29c336 archiver: Add filepath to error message if it is not included yet 2024-01-08 21:33:05 +01:00
Michael Eischer e886c3f6b2 archiver: improve error message for irregular files
Since Go 1.21, most reparse points are considered as irregular files.
Depending on the underlying driver these can exhibit nearly arbitrary
behavior. When encountering such a file, restic returned an
indecipherable error message: `error: invalid node type ""`.

Add the filepath to the error message and state that the file type is
not supported.
2024-01-08 21:33:05 +01:00
Michael Eischer c95de54726 restic: cleanup node type determination
os.ModeCharDevice is already included in os.ModeType
2024-01-08 21:33:05 +01:00
Michael Eischer d4b8abd3e2 fix deduplicated files on windows 2024-01-08 21:33:05 +01:00
Joram Berger 948ab3ccaf Add a note that the oldest snapshot may be kept additionally
Documentation enhancement.
2024-01-08 21:33:05 +01:00
Markus Zoppelt bb0c923298 docs: add pkgx install option
PR in pkgx pantry: https://github.com/pkgxdev/pantry/pull/4098

restic pkg:
https://pkgx.dev/pkgs/restic.net/restic/
2024-01-08 21:33:05 +01:00
Michael Eischer ff0c975443 regenerate changelog 2024-01-08 21:33:05 +01:00
Michael Eischer 7e61e117d6 cleanup changelog whitespace 2024-01-08 21:33:05 +01:00
mmattel 220a28582e Add a table of contents (TOC) to the changelog template 2024-01-08 21:33:05 +01:00
Giuseppe D'Andrea f44fd73230 docs: fix typo in working with repos
When using the `copy` command, `--from-password-file` and `--from-password-command` flags are used to specify the password of the source repository, not of the destination repository.
2024-01-08 21:33:05 +01:00
Joda Stößer 76bd975e03 docs(scripting): correct stats output comment to be about the correct command
not about the snapshots command
2024-01-08 21:33:05 +01:00