restic/internal
Michael Eischer 120ccc8754 repository: Rework blob saving to use an async pack uploader
Previously, SaveAndEncrypt would assemble blobs into packs and either
return immediately if the pack is not yet full or upload the pack file
otherwise. The upload will block the current goroutine until it
finishes.

Now, the upload is done using separate goroutines. This requires changes
to the error handling. As uploads are no longer tied to a SaveAndEncrypt
call, failed uploads are signaled using an errgroup.

To count the uploaded amount of data, the pack header overhead is no
longer returned by `packer.Finalize` but rather by
`packer.HeaderOverhead`. This helper method is necessary to continue
returning the pack header overhead directly to the responsible call to
`repository.SaveBlob`. Without the method this would not be possible,
as packs are finalized asynchronously.
2022-07-02 22:42:34 +02:00
..
archiver repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
backend Merge pull request #3470 from MichaelEischer/sanitize-debug-log 2022-07-02 19:00:54 +02:00
bloblru bloblru: Fix comment for New function 2022-03-28 22:25:25 +02:00
cache cache: Don't Lstat before creating the tag file 2022-05-10 18:52:39 +02:00
checker repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
crypto crypto: Remove unused error 2020-09-05 10:07:16 +02:00
debug redact swift auth token in debug output 2022-07-02 18:47:35 +02:00
dump Refactor internal/dump + concurrent load/write 2021-11-01 23:01:55 +01:00
errors all: Move away from pkg/errors, easy cases 2022-06-14 08:36:38 +02:00
filter Add tests for validating exclude patterns 2022-05-11 22:41:00 +02:00
fs Add simple test for fs.TempFile on windows 2022-04-09 23:37:58 +02:00
fuse copy/find/ls/recover/stats: Memorize snapshot listing before index 2022-04-09 12:26:30 +02:00
hashing hashing: Fix up comments 2022-05-11 21:36:10 +02:00
limiter cmd/restic, limiter: Move config knowledge to internal packages 2022-06-22 18:29:58 +02:00
migrations migrate: Cleanup option to request repository check 2022-06-04 23:45:00 +02:00
mock upgrade_repo_v2: Use atomic replace for supported backends 2022-05-09 22:31:30 +02:00
options Don't crash if SecretString is uninitialized 2022-07-02 19:44:28 +02:00
pack repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
repository repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
restic repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
restorer repository: Rework blob saving to use an async pack uploader 2022-07-02 22:42:34 +02:00
selfupdate Refactor file handing for self-update. 2022-04-09 21:40:33 +02:00
test all: Move away from pkg/errors, easy cases 2022-06-14 08:36:38 +02:00
textfile Add more error handling 2021-01-30 20:19:47 +01:00
ui Print number of bytes added to the repo 2022-07-02 18:55:12 +02:00
walker Limit number of large tree blobs loaded in parallel by StreamTrees 2022-02-19 12:26:09 +01:00