rdelaage
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,886 Commits 7 Branches 44 Tags 74 MiB
Commit Graph

5886 Commits

Author SHA1 Message Date
Michael Eischer 7132df529e repository: Increase index size for repo version 2 
A compressed index is only about one third the size of an uncompressed
one. Thus increase the number of entries in an index to avoid cluttering
the repository with small indexes.
 2022-04-30 11:34:10 +02:00
Michael Eischer 2535524132 debug: Add support for compressed blobs 2022-04-30 11:34:10 +02:00
Michael Eischer fda7bb0f09 debug: Reduce code duplication 2022-04-30 11:34:10 +02:00
Michael Eischer 66f9048bce repository: Alloc zstd encoder/decoder on demand 2022-04-30 11:34:10 +02:00
Michael Eischer fd05037e1a repository: recalibrate index batch allocation size 2022-04-30 11:34:10 +02:00
Michael Eischer 6fb408d90e repository: implement pack compression 2022-04-30 11:34:10 +02:00
Michael Eischer 362ab06023 init: Add flag to specify created repository version 2022-04-30 10:07:42 +02:00
Michael Eischer 4b957e7373 repository: Implement index/snapshot/lock compression 
The config file is not compressed as it should remain readable by older
restic versions such that these can return a proper error.

As the old format for unpacked data does not include a version header,
make use of a trick: The old data is always encoded as JSON. Thus it can
only start with '{' or '['. For any other value the first byte indicates
a versioned format. The version is set to 2 for now. Then the zstd
compressed data follows.
 2022-04-30 10:07:42 +02:00
Michael Eischer 0957b74887 Misc design.rst cleanups 2022-04-30 10:07:42 +02:00
Alexander Neumann 270ed00d1f doc: Add repository compression support documentation 
Co-authored-by: Michael Eischer <michael.eischer@fau.de>
 2022-04-30 10:07:42 +02:00
Alexander Neumann 4e1ef7804a
Merge pull request #3717 from MichaelEischer/fix-stuck-repack 
Fix stuck repack step
 2022-04-30 09:50:43 +02:00
Alexander Neumann e4780d3956
Merge pull request #3718 from MichaelEischer/sftp-docs-fix 
doc: sftp with password actually works
 2022-04-23 20:27:27 +02:00
Alexander Neumann c183e35b5a
Merge pull request #3719 from MichaelEischer/read-write-order 
Describe repository read/write order required by repository format
 2022-04-23 15:23:00 +02:00
Michael Eischer 6f9e20a1bb doc: Describe repository read/write order 2022-04-23 15:21:02 +02:00
Michael Eischer f9219e8608 doc: sftp with password actually works 2022-04-23 11:57:36 +02:00
Michael Eischer 3b630d9998 add missing streamPacks changelog 2022-04-23 11:50:19 +02:00
Michael Eischer 566ac11c65 fix changelog name 2022-04-23 11:37:00 +02:00
Michael Eischer f5609d1d3c prune: Fail early if too few backend connections 2022-04-23 11:32:52 +02:00
Michael Eischer e597b99b55 repository: Reduce repack workers to prevent deadlock 
As repack streams packs these occupy one backend connection. Uploading a
new pack also requires a backend connection. To prevent a deadlock
during repack when reaching the backend connections limit, simply limit
the repackWorker count to always leave one connection for uploading.
 2022-04-23 11:28:18 +02:00
Michael Eischer ee627cd832 backend/mem: Actually enforce connection limit 
This will allow tests to detect deadlocks related to the connections
limit.
 2022-04-23 11:22:00 +02:00
Michael Eischer 4f97492d28 Backend: Expose connections parameter 2022-04-23 11:13:08 +02:00
rawtaz 07a565e6f7
Merge pull request #3716 from MichaelEischer/password-error-on-stderr 
Print password error message on stderr
 2022-04-21 01:40:54 +02:00
Michael Eischer bf7da7ff10 Print password error message on stderr 
The password prompt itself is already printed on stderr.
 2022-04-20 22:22:09 +02:00
Alexander Neumann dba47d29d5
Merge pull request #3711 from restic/doc-grouping 
doc: Clarify and make grouping in forget more noticeable
 2022-04-16 10:08:11 +02:00
Leo R. Lundgren 8ac7519fd5 doc: Clarify and make grouping in forget more noticeable 2022-04-16 01:13:13 +02:00
Alexander Neumann edc1a24a90
Merge pull request #3707 from duracell/patch-1 
doc: fix missing "init" in rest-server example
 2022-04-12 20:39:25 +02:00
Michael 9563e2f75c
doc: fix missing "init" in rest-server example 2022-04-12 18:21:53 +02:00
Alexander Neumann 7f133a28b2 Update VERSION file 2022-04-11 20:34:14 +02:00
Alexander Neumann 4f3b1f19cb Set development version for 0.13.1 2022-04-10 20:41:00 +02:00
Alexander Neumann 89ee1cf9ee
Merge pull request #3610 from MichaelEischer/windows-temp-files 
Improve handling of temporary files on windows
 2022-04-10 20:29:04 +02:00
Alexander Neumann a059ef90f8
Merge pull request #3702 from MichaelEischer/extend-config-error 
Print used key name if config fails to load
 2022-04-10 20:25:24 +02:00
Michael Eischer 4077a81b34 Add simple test for fs.TempFile on windows 2022-04-09 23:37:58 +02:00
Michael Eischer 9a3f1a9703 Simplify and comment TempFile implementation for windows 2022-04-09 23:37:58 +02:00
Michael Eischer c2aabb2686 Print used key name if config fails to load 2022-04-09 22:38:18 +02:00
MichaelEischer c60a5f00c9
Merge pull request #3675 from ItsMattL/update 
Refactor file handing for self-update.
 2022-04-09 21:55:56 +02:00
Matt LaPlante 0ba9d4ced7 Refactor file handing for self-update. 
* Write new file payload to a temp file before touching the original
binary. Minimizes the possibility of failing mid-write and corrupting
the binary.
* On Windows, move the original binary out to a temp file rather than
removing it as the running binary is locked. Fixes issue #2248.
 2022-04-09 21:40:33 +02:00
Alexander Neumann 04e054465a
Merge pull request #3475 from MichaelEischer/local-sftp-conn-limit 
Limit concurrent operations for local / sftp backend
 2022-04-09 21:33:00 +02:00
Alexander Neumann 1519e9f911
Merge pull request #3570 from MichaelEischer/list-snapshots-before-index 
List snapshots before index
 2022-04-09 21:15:49 +02:00
Michael Eischer ebab35581c Check in integration test that snapshots are listed before the index 
As an exception prune is still allowed to load the index before
snapshots, as it uses exclusive locks. In case of problems with locking
it is also better to load snapshots created after loading the index, as
this will lead to a prune sanity check failure instead of a broken snapshot.
 2022-04-09 12:27:27 +02:00
Michael Eischer 7b9ae91e04 copy: Load snapshots before indexes 2022-04-09 12:27:25 +02:00
Michael Eischer 47243176fa diff: list snapshots only once 2022-04-09 12:26:31 +02:00
Michael Eischer 5af828e3e6 add changelogs 2022-04-09 12:26:31 +02:00
Michael Eischer 4636c20397 test that TestFindListOnce calls List only once 2022-04-09 12:26:31 +02:00
Michael Eischer 9e12159230 Fix O(n) backend list calls in FindFilteredSnapshots 
When resolving snapshotIDs in FindFilteredSnapshots either
FindLatestSnapshot or FindSnapshot is called. Both operations issue a
list operation to the backend. When for example passing a long list of
snapshot ids to `forget` this could lead to a large number of list
operations.
 2022-04-09 12:26:31 +02:00
Michael Eischer 3d29083e60 copy/find/ls/recover/stats: Memorize snapshot listing before index 
These commands filter the snapshots according to some criteria which
essentially requires loading the index before filtering the snapshots.
Thus create a copy of the snapshots list beforehand and use it later on.
 2022-04-09 12:26:30 +02:00
Michael Eischer 2ec0f3303a backup/diff/dump/restore/stats: List snapshots before index 
During a backup the index is written before the corresponding snapshots.
To ensure that a concurrent/later restic run can read a snapshot's data,
restic thus must first load the snapshots and only afterwards the index.
Otherwise it is not possible to ensure that the loaded index is recent
enough to cover all of the snapshot's data.
 2022-04-09 12:24:09 +02:00
Michael Eischer ece06f125e sftp: Limit concurrent backend operations 2022-04-09 12:21:38 +02:00
Michael Eischer cd783358d3 local: Limit concurrent backend operations 
Use a limit of 2 similar to the filereader concurrency in the archiver.
 2022-04-09 12:21:38 +02:00
Michael Eischer 0b258cc054 backends: clean reader closing 2022-04-09 12:21:38 +02:00
Alex Duchesne 9e34c791c9 Better temp file cleanup on Windows. 2022-04-09 12:00:22 +02:00