rdelaage
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,773 Commits 7 Branches 44 Tags 74 MiB
Commit Graph

340 Commits

Author SHA1 Message Date
Michael Eischer 10355c3fb6 repository: Better error message if blob is larger than 4GB 2024-04-19 22:00:35 +02:00
Michael Eischer 5e98f1e2eb repository: fix test setup race conditions 2024-03-28 23:17:02 +01:00
Michael Eischer dc441c57a7 repository: unify repository initialization in tests 
Tests should use a helper from internal/repository/testing.go to
construct a Repository object.
 2024-03-28 23:17:02 +01:00
Michael Eischer 3ba1fa3cee repository: remove a few global variables 2024-03-28 23:17:02 +01:00
Michael Eischer 044e8bf821 repository: parallelize lock tests 2024-03-28 23:17:02 +01:00
Michael Eischer e8df50fa3c repository: remove global list of locks 2024-03-28 22:46:33 +01:00
Michael Eischer cbb5f89252 lock: move code to repository package 2024-03-28 22:46:33 +01:00
Michael Eischer 4c3218ef9f repository: include packID in StreamPack for decrypt/decompress errors 2024-02-17 19:38:01 +01:00
Michael Eischer 18b0bbbf42 repository: use fmt.Errorf in StreamPacks 2024-02-17 19:37:32 +01:00
Alexander Neumann c0514dd8ba Fix linter errors (except for tests) 2024-02-10 22:58:10 +01:00
Michael Eischer 5957417b1f Apply changelog entry / documentation improvements from review 2024-02-04 18:55:41 +01:00
Michael Eischer 86b38a0b17 rename `--no-verify-pack` to `--no-extra-verify` 2024-02-04 17:01:05 +01:00
Michael Eischer c97a271e89 repository: ask users to report corrupted data while saving blobs 2024-02-04 15:31:42 +01:00
Michael Eischer 193140525c repository: test verification of blobs/unpacked data 2024-02-04 15:31:42 +01:00
Michael Eischer 2dbb18128c repository: Allow skipping verification for tests 
Some tests have to explicitly create pack files with blobs that don't
match their ID. For those blobs the builtin verification of the
repository must be disabled.
 2024-02-03 18:22:47 +01:00
Michael Eischer 30a84e9003 backup: verify unpacked files before upload 2024-02-03 18:22:47 +01:00
Michael Eischer c01a0c6da7 backup: verify blobs before upload 
This only covers the blobs themselves, the pack header is not verified
so far. Unpacked files are also not covered by the integrity check.
 2024-02-03 18:22:47 +01:00
Michael Eischer 16e3f79e8b repository: make repo.Options configurable for test repos 2024-02-03 18:22:47 +01:00
Michael Eischer bb92b487f7 repository: fix repack test 2024-02-03 18:22:47 +01:00
Michael Eischer bfb56b78e1 replace some usages of restic.Repository with more specific interface 
This should eventually make it easier to test the code.
 2024-01-27 13:02:02 +01:00
Michael Eischer 3424088274
Merge pull request #4644 from MichaelEischer/refactor-repair-packs 
Refactor and test `repair packs`
 2024-01-27 13:00:51 +01:00
Michael Eischer f0e1ad2285 fix linter warning 2024-01-27 12:51:45 +01:00
Michael Eischer fd579421dd repository: deduplicate test 2024-01-27 12:51:45 +01:00
Michael Eischer 42c9318b9c repair pack: add tests 2024-01-27 12:51:45 +01:00
Michael Eischer 764b0bacd6 repair pack: add support for truncated files 2024-01-27 12:51:45 +01:00
Michael Eischer 7c351bc53c repair pack: reenable auto index updates 
The method is not available on the restic.Repository interface that is
used for testing. Drop the call as a small amount of additional index
writes is not a problem.
 2024-01-27 12:51:45 +01:00
Michael Eischer feeab84204 repair pack: extract the repair logic into the repository package 
Currently, the cmd/restic package contains a significant amount of code
that modifies repository internals. This code should in the mid-term
move into the repository package.
 2024-01-27 12:51:45 +01:00
Michael Eischer cb50832d50 index: let MasterIndex.Save also delete obsolete indexes 2024-01-27 12:51:08 +01:00
Michael Eischer c13bf0b607 repository: Introduce RemoveKey function 
This replaces directly removing keys via the backend.
 2024-01-27 12:42:58 +01:00
Michael Eischer 2c310a526e repository: Replace StreamPack function with LoadBlobsFromPack method 
LoadBlobsFromPack is now part of the repository struct. This ensures
that users of that method don't have to deal will internals of the
repository implementation.

The filerestorer tests now also contain far fewer pack file
implementation details.
 2024-01-19 21:40:43 +01:00
Michael Eischer 6b7b5c89e9 repository: prepare StreamPack refactor 2024-01-19 21:40:43 +01:00
Michael Eischer fb422497af repository: split StreamPack implementation 
Move the actual decoding of the pack data into a separate iterator.
 2024-01-19 21:39:55 +01:00
Michael Eischer 77b1c52673 repository: test that StreamPack only delivers blobs once 2024-01-07 10:54:53 +01:00
Michael Eischer fe5c337ca2 repository: StreamPack delivers blobs at most once 
If an error occurred while streaming a pack file, this could result in
passing some of the blobs multiple times to the callback function. This
significantly complicates using StreamPack correctly and is unnecessary.
Retries do not change the content of a blob and thus only deliver the
same result over and over again.
 2024-01-07 10:54:49 +01:00
Andrea Gelmini 241916d55b
Fix typos 2023-12-06 13:11:55 +01:00
Michael Eischer 45962c2847
Merge pull request #4499 from MichaelEischer/modular-backend-code 
Split backend code from restic package
 2023-10-27 20:19:20 +02:00
Leo R. Lundgren aafb806a8c doc: Correct two typos 2023-10-27 18:56:32 +02:00
Michael Eischer c7b770eb1f convert MemorizeList to be repository based 
Ideally, code that uses a repository shouldn't directly interact with
the underlying backend. Thus, move MemorizeList one layer up.
 2023-10-25 23:01:35 +02:00
Michael Eischer 1b8a67fe76 move Backend interface to backend package 2023-10-25 23:00:18 +02:00
Michael Eischer b6d79bdf6f restic: decouple restic.Handle 2023-10-25 22:54:07 +02:00
Michael Eischer cb9cbe55d9 repository: store oversized blobs in separate pack files 
Store oversized blobs in separate pack files as the blobs is large
enough to warrant its own pack file. This simplifies the garbage
collection of such blobs and keeps the cache smaller, as oversize (tree)
blobs only have to be downloaded if they are actually used.
 2023-10-17 22:52:16 +02:00
Michael Eischer 3fd0ad7448 repository: list index files only once 2023-10-01 19:53:26 +02:00
arjunajesh ed65a7dbca implement progress bar for index loading 2023-10-01 19:52:59 +02:00
Michael Eischer 191c47d30e
Merge pull request #4353 from MichaelEischer/tune-gc 
Tune Go garbage collector
 2023-06-16 23:24:39 +02:00
Michael Eischer eef0ee7a85 repository: trigger GC after loading the index 
Loading the index requires some scratch space, thus make sure that this
memory does not factor into the targeted gc memory usage limit.
 2023-06-02 21:56:14 +02:00
Michael Eischer ffca602315 repository: Fix panic in benchmarkLoadIndex 2023-05-28 23:55:47 +02:00
Michael Eischer d1a5ec7839 Rename unused testing parameter to _ 
The parameter is an additional marker that the test helper must only be
used for tests.
 2023-05-18 21:17:53 +02:00
Michael Eischer 1514593f22 Remove unused context or testing parameters 2023-05-18 21:17:53 +02:00
Michael Eischer e01baeabba Use either test or rtest to refer to internal test helpers 
A single test file should not use both names.
 2023-05-18 21:15:45 +02:00
Michael Eischer 5773b86d02 repository: Push all usage of errors.Fatal out of the package 
As the `Fatal` error type only includes a string, it becomes impossible
to inspect the contained error. This is for a example a problem for the
fuse implementation, which must be able to detect context.Canceled
errors.

Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
 2023-05-18 17:27:41 +02:00