Michael Eischer
8898f61717
Merge pull request #4809 from MichaelEischer/update-changelog
...
add retries for corrupted blobs to changelog
2024-05-18 23:04:13 +02:00
Michael Eischer
5f23baabcc
add retries for corrupted blobs to changelog
2024-05-18 23:03:24 +02:00
Michael Eischer
9c5bac6f25
Merge pull request #4799 from letmaik/letmaik/azure-force-cli-credential
...
Azure: add option to force use of CLI credential
2024-05-18 20:22:15 +00:00
Michael Eischer
c56ecec9aa
azure: deduplicate cli and default credentials case
2024-05-18 22:15:54 +02:00
Maik Riechert
355f520936
Azure: add option to force use of CLI credential
2024-05-18 22:15:54 +02:00
Michael Eischer
1dfe1b8732
Merge pull request #4802 from MichaelEischer/backend-cleanups
...
Repository: Remove Backend() method
2024-05-18 22:02:45 +02:00
Michael Eischer
223aa22cb0
replace some uses of restic.Repository with finegrained interfaces
2024-05-18 21:42:51 +02:00
Michael Eischer
291c9677de
restic/repository: remove Backend() method
2024-05-18 21:42:51 +02:00
Michael Eischer
673496b091
repository: clean cache between CheckPack retries
...
The cache cleanup pattern is also used in ListPack etc.
2024-05-18 21:42:51 +02:00
Michael Eischer
3d2410ed50
Replace some repo.RemoveUnpacked usages
...
These will eventually be blocked as they do not delete Snapshots.
2024-05-18 21:42:51 +02:00
Michael Eischer
d2c26e33f3
repository: remove further usages of repo.Backend()
2024-05-18 21:42:51 +02:00
Michael Eischer
8a425c2f0a
remove usages of repo.Backend() from tests
2024-05-18 21:42:51 +02:00
Michael Eischer
aa4647f773
repository: unexport PackBlobIterator
2024-05-18 21:42:51 +02:00
Michael Eischer
94e863885c
check: move verification of individual pack file to repository
2024-05-18 21:42:50 +02:00
Michael Eischer
e40943a75d
restic: remove backend usage from lock test
2024-05-18 21:38:31 +02:00
Michael Eischer
67e2ba0d40
repository: Lock requires *repository.Repository
...
This allows the Lock function to access the backend, even once the
Backend method is removed from the interface.
2024-05-18 21:38:31 +02:00
Michael Eischer
d8b184b3d3
repository: convert test helper to return *repository.Repository
2024-05-18 21:38:31 +02:00
Michael Eischer
a1ca5e15c4
migrations: add temporary hack for s3_layout
...
The migration will be removed after the next restic release anyways.
Thus, there's no need for a clean implementation.
2024-05-18 21:38:31 +02:00
Michael Eischer
34d90aecf9
migrations: move logic of upgrade_repo_v2 to repository package
...
The migration modifies repository internals and thus should live within
the repository package.
2024-05-18 21:38:31 +02:00
Michael Eischer
ab9077bc13
replace usages of backend.Remove() with repository.RemoveUnpacked()
...
RemoveUnpacked will eventually block removal of all filetypes other than
snapshots. However, getting there requires a major refactor to provide
some components with privileged access.
2024-05-18 21:38:31 +02:00
Michael Eischer
8274f5b101
prune: remove Backend.IsNotExist()
...
Only handling one specific error is not particularly useful.
2024-05-18 21:38:31 +02:00
Michael Eischer
9795198189
debug: remove Backend.Stat() usage
2024-05-18 21:38:31 +02:00
Michael Eischer
0c1ba6d95d
backend: remove unused Location method
2024-05-18 21:38:31 +02:00
Michael Eischer
eb6c653f89
Merge pull request #4800 from MichaelEischer/cleanup-load
...
Retry loading of corrupted data from backend / cache
2024-05-18 21:34:54 +02:00
Michael Eischer
74d90653e0
check: use ReadFull to load pack header in checkPack
...
This ensures that the pack header is actually read completely.
Previously, for a truncated file it was possible to only read a part of
the header, as backend.Load(...) is not guaranteed to return as many
bytes as requested by the length parameter.
2024-05-18 21:28:54 +02:00
Michael Eischer
8f8d872a68
fix compatibility with go 1.19
2024-05-18 21:28:54 +02:00
Michael Eischer
ff0744b3af
check: test checkPack retries
2024-05-18 21:28:54 +02:00
Michael Eischer
987c3b250c
repository: test retries of ListPack
2024-05-18 21:28:54 +02:00
Michael Eischer
bf16096771
repository: test LoadBlob retries
2024-05-18 21:28:54 +02:00
Michael Eischer
4f45668b7c
repository: rework and extend LoadRaw tests
2024-05-18 21:28:54 +02:00
Michael Eischer
ac805d6838
cache: cleanup debug logs
2024-05-18 21:28:54 +02:00
Michael Eischer
5214af88e2
cache: test forget behavior
2024-05-18 21:28:54 +02:00
Michael Eischer
3ff063e913
check: verify pack a second time if broken
2024-05-18 21:28:54 +02:00
Michael Eischer
385cee09dc
repository: fix caching of tree packs in LoadBlobsFromPack
2024-05-18 21:28:54 +02:00
Michael Eischer
e734746f75
cache: forget cached file at most once
...
This is inspired by the circuit breaker pattern used for distributed
systems. If too many requests fails, then it is better to immediately
fail new requests for a limited time to give the backend time to
recover.
By only forgetting a file in the cache at most once, we can ensure that
a broken file is only retrieved once again from the backend. If the file
stored there is broken, previously it would be cached and deleted
continuously. Now, it is retrieved only once again, all later requests
just use the cached copy and either succeed or fail immediately.
2024-05-18 21:28:54 +02:00
Michael Eischer
97a307df1a
cache: Always use cached file if it exists
...
A file is always cached whole. Thus, any out of bounds access will also
fail when directed at the backend. To handle case in which the cached
file is broken, then caller must call Cache.Forget(h) for the file in
question.
2024-05-18 21:28:54 +02:00
Michael Eischer
8cce06d915
repair packs: drop experimental warning
...
This warning should already have been removed once the feature flag was
dropped.
2024-05-18 21:28:54 +02:00
Michael Eischer
433a6aad29
repository: remove redundant blob loading fallback from RepairPacks
...
LoadBlobsFromPack already implements the same fallback behavior.
2024-05-18 21:28:54 +02:00
Michael Eischer
e401af07b2
check: fix error message formatting
2024-05-18 21:28:54 +02:00
Michael Eischer
7017adb7e9
repository: retry failed ListPack once
2024-05-18 21:28:54 +02:00
Michael Eischer
e33ce7f408
repository: retry failed LoadBlob once
2024-05-18 21:28:54 +02:00
Michael Eischer
2ace242f36
repository: make reloading broken files explicit
2024-05-18 21:28:54 +02:00
Michael Eischer
e9390352a7
cache: code cleanups
2024-05-18 21:26:00 +02:00
Michael Eischer
503c8140b1
repository: unify blob decoding code
2024-05-18 21:26:00 +02:00
Michael Eischer
6563f1d2ca
repository: remove redundant debug log
2024-05-18 21:26:00 +02:00
Michael Eischer
021fb49559
repository: Implement repository.LoadUnpacked using LoadRaw
...
Both functions were using a similar implementation.
2024-05-18 21:26:00 +02:00
Michael Eischer
779c8d3527
debug/repair packs/upgrade repo v2: use repository.LoadRaw
...
This replaces calling the low-level backend.Load() method.
2024-05-18 21:26:00 +02:00
Michael Eischer
1d6d3656b0
repository: move backend.LoadAll to repository.LoadRaw
...
LoadRaw also includes improved context cancellation handling similar to the
implementation in repository.LoadUnpacked.
The removed cache backend test will be added again later on.
2024-05-18 21:26:00 +02:00
Michael Eischer
47232bf8b0
backend: move LimitReadCloser to util package
...
The helper is only intended for usage by backend implementations.
2024-05-18 21:26:00 +02:00
Michael Eischer
dcd151147c
Merge pull request #4803 from restic/permanent-retry-failure
...
Do not retry permanent backend failures
2024-05-18 20:07:06 +02:00
Michael Eischer
53d15bcd1b
retry: add circuit breaker to load method
...
If a file exhausts its retry attempts, then it is likely not accessible
the next time. Thus, immediately fail all load calls for that file to
avoid useless retries.
2024-05-18 19:59:26 +02:00
Michael Eischer
394c8ca3ed
rest/rclone/s3/sftp/swift: move short file detection behind feature gate
...
These backends tend to use a large variety of server implementations.
Some of those implementations might prove problematic with the new
checks.
2024-05-18 19:59:26 +02:00
Michael Eischer
6328b7e1f5
replace "too small" with "too short" in error messages
2024-05-18 19:59:26 +02:00
Michael Eischer
53561474d9
update changelog with persistent backend error handling
2024-05-18 19:59:26 +02:00
Michael Eischer
aeb7eb245c
retry: do not retry permanent errors
...
This is currently gated behind a feature flag as some unexpected
interactions might show up in the wild.
2024-05-18 19:59:26 +02:00
Michael Eischer
bf8cc59889
Use generic backend-error-redesign feature flag instead of http-timeouts
...
An individual flag for each change of the backend error handling would
be too finegrained. Thus, add a generic flag.
2024-05-18 19:54:52 +02:00
Michael Eischer
4740528a0b
backend: add tests for IsPermanentError
2024-05-18 19:54:52 +02:00
Michael Eischer
6a85df7297
backend: add IsPermanentError() method to interface
2024-05-18 19:54:52 +02:00
Michael Eischer
cfc420664a
mem: stricter handling of out of bounds requests
2024-05-18 19:54:52 +02:00
Michael Eischer
d40f23e716
azure/b2/gs/s3/swift: adapt cloud backend
2024-05-18 19:54:51 +02:00
Michael Eischer
e793c002ec
local: stricter handling of short files
2024-05-18 19:54:21 +02:00
Michael Eischer
b4895ebd76
rest: rework error reporting and report too short files
2024-05-18 19:54:21 +02:00
Michael Eischer
eaa3f81d6b
sftp: check for truncated files without an extra backend request
2024-05-18 19:54:21 +02:00
Michael Eischer
c6d74458ee
sftp: improve handling of too short files
2024-05-18 19:54:21 +02:00
Michael Eischer
7ed560a201
Merge pull request #4796 from MichaelEischer/parallel-dump-load
...
dump: Parallelize loading large files
2024-05-14 22:35:44 +02:00
Michael Eischer
92221c2a6d
Merge pull request #4708 from zmanda/windows-securitydesc
...
Back up and restore SecurityDescriptors on Windows
2024-05-12 14:14:39 +00:00
Michael Eischer
b5fdb1d637
Merge pull request #4782 from MichaelEischer/fix-sftp-performance
...
Fix sftp upload performance
2024-05-12 15:28:33 +02:00
Michael Eischer
e4f9bce384
Merge pull request #4792 from restic/request-watchdog
...
backend: enforce that backend HTTP requests make progress
2024-05-09 23:55:30 +02:00
Michael Eischer
3740700ddc
add http timeouts to changelog
2024-05-09 23:46:17 +02:00
Michael Eischer
ebd01a4675
backend: add tests for watchdogRoundTripper
2024-05-09 23:46:17 +02:00
Michael Eischer
8778670232
backend: cancel stuck http requests
...
requests that make no upload or download progress within a timeout are
canceled.
2024-05-09 23:46:17 +02:00
Michael Eischer
0987c731ec
backend: configure protocol-level connection health checks
...
This should detect a connection that is stuck for more than 2 minutes.
2024-05-09 23:46:17 +02:00
aneesh-n
a4fd1b91e5
Fix review comments
...
Change lowerPrivileges from bool to atomic.Bool.
Add missing cleanup from upstream go-winio.
Add handling for ERROR_NOT_ALL_ASSIGNED warning.
2024-05-06 16:54:08 -06:00
Michael Eischer
e184538ddf
dump: add changelog
2024-05-05 12:12:21 +02:00
Michael Eischer
4d55a62ada
bloblru: add test for GetOrCompute
2024-05-05 12:00:25 +02:00
Michael Eischer
7cce667f92
fuse: switch to use bloblru.GetOrCompute
2024-05-05 11:38:42 +02:00
Michael Eischer
bd03af2feb
dump: add GetOrCompute to bloblru cache
2024-05-05 11:38:42 +02:00
Michael Eischer
45509eafc8
dump: load blobs of a file from repository in parallel
2024-05-05 11:38:42 +02:00
Michael Eischer
24c1822220
Merge pull request #4794 from flow-c/master
...
Update 060_forget.rst
2024-05-04 08:25:06 +00:00
flow-c
d4477a5a99
Update 060_forget.rst
...
Replace deprecated `-1` with `unlimited` in calendar-related `--keep-*` options
2024-05-04 09:32:25 +02:00
Michael Eischer
ffe5439149
Merge pull request #4605 from MichaelEischer/better-restorer-error-handling
...
Rework repository.StreamPacks & better restorer error handling
2024-05-01 16:37:41 +02:00
Michael Eischer
676f0dc60d
add changelog
2024-05-01 16:28:57 +02:00
Michael Eischer
1e57057953
Merge pull request #4789 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob-1.3.2
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 1.3.1 to 1.3.2
2024-05-01 10:45:47 +00:00
Michael Eischer
1ba0af6993
Merge pull request #4787 from restic/dependabot/go_modules/github.com/klauspost/compress-1.17.8
...
build(deps): bump github.com/klauspost/compress from 1.17.7 to 1.17.8
2024-05-01 10:44:33 +00:00
Michael Eischer
ffc41ae62a
Merge pull request #4786 from restic/dependabot/go_modules/golang.org/x/net-0.24.0
...
build(deps): bump golang.org/x/net from 0.23.0 to 0.24.0
2024-05-01 10:41:26 +00:00
Michael Eischer
4832c2fbfa
Merge pull request #4790 from restic/dependabot/github_actions/golangci/golangci-lint-action-5
...
build(deps): bump golangci/golangci-lint-action from 4 to 5
2024-05-01 10:37:37 +00:00
dependabot[bot]
30609ae6b2
build(deps): bump golangci/golangci-lint-action from 4 to 5
...
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action ) from 4 to 5.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases )
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v4...v5 )
---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-05-01 01:45:43 +00:00
dependabot[bot]
502e5867a5
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go ) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.3.1...sdk/storage/azblob/v1.3.2 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-05-01 01:02:39 +00:00
dependabot[bot]
18a6d6b408
build(deps): bump github.com/klauspost/compress from 1.17.7 to 1.17.8
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.17.7 to 1.17.8.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.17.7...v1.17.8 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-05-01 01:02:22 +00:00
dependabot[bot]
3bb88e8307
build(deps): bump golang.org/x/net from 0.23.0 to 0.24.0
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.23.0 to 0.24.0.
- [Commits](https://github.com/golang/net/compare/v0.23.0...v0.24.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-05-01 01:02:11 +00:00
aneesh-n
672f6cd776
Fix review comments for privileges and security flags
2024-04-29 17:29:51 -06:00
aneesh-n
08c6945d61
Fix review comments
2024-04-29 16:21:38 -06:00
Aneesh N
3f76b902e5
Merge branch 'master' into windows-securitydesc
2024-04-29 14:40:34 -06:00
Michael Eischer
ccac7c7fb3
Merge pull request #3067 from DRON-666/vss-options
...
Add options to fine tune VSS snapshots
2024-04-29 18:09:47 +00:00
DRON-666
ccd35565ee
s/sec./seconds
2024-04-29 01:48:22 +03:00
DRON-666
125dba23c5
Rearange code
2024-04-29 01:27:34 +03:00
DRON-666
7ee889bb0d
Use S_FALSE and MaxInt
2024-04-29 01:25:25 +03:00
DRON-666
90b168eb6c
isMountPointExcluded to isMountPointIncluded
2024-04-29 01:23:50 +03:00
DRON-666
24330c19a8
Use kebab case in option names
2024-04-29 01:21:33 +03:00
DRON-666
5703e5a652
Fix texts and comments
2024-04-29 01:18:46 +03:00
DRON-666
0a8f9c5d9c
vss: Add tests for "provider" option
2024-04-28 22:45:21 +03:00
DRON-666
739d3243d9
vss: Update docs and changelog
2024-04-28 22:45:21 +03:00
DRON-666
bb0f93ef3d
vss: Add "provider" option
2024-04-28 22:45:21 +03:00
DRON-666
3bac1f0135
vss: Fix issues reported by linters
2024-04-28 22:45:21 +03:00
DRON-666
88c509e3e9
vss: Change `ErrorHandler` signature
...
We don't need `error` here: the only existing implementation
of `ErrorHandler` always call `Backup.Error` and all
implementations of `Backup.Error` always return nil.
2024-04-28 22:44:16 +03:00
DRON-666
9d3d915e2c
vss: Add some tests
2024-04-28 22:44:16 +03:00
DRON-666
9182e6bab5
vss: Update docs and changelog
2024-04-28 22:44:16 +03:00
DRON-666
c4f67c0064
vss: Add volume filtering
...
Add options to exclude all mountpoints and arbitrary volumes from snapshotting.
2024-04-28 22:44:15 +03:00
DRON-666
7470e5356e
vss: Add "timeout" option
...
Changing multiple "callAsyncFunctionAndWait" with fixed timeout
to calculated timeout based on deadline.
2024-04-28 22:44:15 +03:00
DRON-666
78dbc5ec58
vss: Add initial support for extended options
2024-04-28 22:44:15 +03:00
Michael Eischer
a1d682ce0e
add changelog for sftp performance fix
2024-04-28 11:58:08 +02:00
Michael Eischer
935327d480
sftp: slightly increase write concurrency
...
This should increase upload throughput for high latency links a bit.
2024-04-28 11:50:09 +02:00
Michael Eischer
669a669603
sftp: Fix upload performance issue
...
Since pkg/sftp 1.13.0 files were uploaded sequentially using 32kb chunks
instead of sending 64 chunks in parallel.
2024-04-28 11:48:26 +02:00
Michael Eischer
faffd15d13
Merge pull request #4734 from maouw/enhancement/envvar-for-host
...
Add support for specifying --host via environment variable
2024-04-24 20:00:15 +00:00
Michael Eischer
347e9d0765
complete RESITC_HOST environment handling & test
2024-04-24 21:52:39 +02:00
Altan Orhon
871ea1eaf3
Add support for specifying --host via environment variable
...
This commit adds support for specifying the `--host` option via the `RESTIC_HOST` environment variable. This is done by extending option processing in `cmd_backup.go` and for `restic.SnapshotFilter` in `find.go`.
2024-04-24 21:49:42 +02:00
Michael Eischer
a7b5e09902
Merge pull request #4753 from MichaelEischer/remove-cleanup-handlers
...
Replace cleanup handlers with context based command cancelation
2024-04-24 21:34:19 +02:00
Michael Eischer
3f9d50865d
Merge pull request #4776 from MichaelEischer/cleanup-backend-open
...
unify backend open and create
2024-04-24 21:24:27 +02:00
Michael Eischer
5f263752d7
init: also apply limiter for non-HTTP backend
2024-04-24 20:42:30 +02:00
Michael Eischer
484dbb1cf4
get rid of a few global variables
2024-04-22 22:39:33 +02:00
Michael Eischer
940a3159b5
let index.Each() and pack.Size() return error on canceled context
...
This forces a caller to actually check that the function did complete.
2024-04-22 22:39:32 +02:00
Michael Eischer
31624aeffd
Improve command shutdown on context cancellation
2024-04-22 22:31:38 +02:00
Michael Eischer
910927670f
mount: fix exit code on cancellation
2024-04-22 22:27:19 +02:00
Michael Eischer
6f2a4dea21
remove global shutdown hook
2024-04-22 22:27:19 +02:00
Michael Eischer
699ef5e9de
debug: replace cleanup handler usage in profiling setup
2024-04-22 22:27:19 +02:00
Michael Eischer
eb710a28e8
use standalone shutdown hook for readPasswordTerminal
...
move terminal restoration into readPasswordTerminal
2024-04-22 22:27:19 +02:00
Michael Eischer
86c7909f41
mount: use standalone shutdown hook via goroutine
2024-04-22 22:27:19 +02:00
Michael Eischer
93135dc705
lock: drop cleanup handler
2024-04-22 22:27:19 +02:00
Michael Eischer
21a7cb405c
check: replace cleanup handler
2024-04-22 22:27:19 +02:00
Michael Eischer
b15d867414
Merge pull request #4763 from MichaelEischer/refactor-prune
...
Refactor repair index / prune into the repository package
2024-04-22 22:24:53 +02:00
Michael Eischer
2e6c43c695
Merge pull request #4761 from MichaelEischer/fix-cache-race
...
cache: ignore ErrNotExist during cleanup of old files
2024-04-22 21:46:06 +02:00
Michael Eischer
f7632de3d6
Merge pull request #4772 from MichaelEischer/better-error-on-too-large-blob
...
repository: Better error message if blob is larger than 4GB
2024-04-22 21:45:06 +02:00
Michael Eischer
20d8eed400
repository: streamPack: separate requests for gap larger than 1MB
...
With most cloud providers, traffic is much more expensive than API
calls. Thus slightly bias streamPack towards a bit more API calls in
exchange for slightly less traffic.
2024-04-22 21:21:23 +02:00
Michael Eischer
cf700d8794
repository: streamPack: reuse zstd decoder
2024-04-22 21:21:23 +02:00
Michael Eischer
666a0b0bdb
repository: streamPack: replace streaming with chunked download
...
Due to the interface of streamPack, we cannot guarantee that operations
progress fast enough that the underlying connections remains open. This
introduces partial failures which massively complicate the error
handling.
Switch to a simpler approach that retrieves the pack in chunks of 32MB.
If a blob is larger than this limit, then it is downloaded separately.
To avoid multiple copies in memory, an auxiliary interface
`discardReader` is introduced that allows directly accessing the
downloaded byte slices, while still supporting the streaming used by the
`check` command.
2024-04-22 21:21:23 +02:00
Michael Eischer
621012dac0
repository: Add blob loading fallback to LoadBlobsFromPack
...
Try to retrieve individual blobs via LoadBlob if streaming did not work.
2024-04-21 21:35:55 +02:00
Michael Eischer
6c6dceade3
global: unify backend open and create
2024-04-19 22:26:14 +02:00
Michael Eischer
10355c3fb6
repository: Better error message if blob is larger than 4GB
2024-04-19 22:00:35 +02:00
Michael Eischer
228b35f074
Merge pull request #4769 from will-ca/patch-1
...
Tiny wording clarification in `restic-stats.1`.
2024-04-18 19:00:39 +00:00
will-ca
6aced61c72
Tiny docs wording clarification.
2024-04-18 07:29:55 +00:00
Michael Eischer
4d22412e0c
Merge pull request #4766 from coderwander/master
...
Fix struct names
2024-04-18 06:18:19 +00:00
coderwander
a82ed71de7
Fix struct names
...
Signed-off-by: coderwander <770732124@qq.com>
2024-04-18 10:02:09 +08:00
Michael Eischer
2173c69280
Merge pull request #4770 from testwill/close_files
...
fix: close files
2024-04-17 16:50:20 +00:00
Michael Eischer
001bb71676
repair packs: Properly close backup files
2024-04-17 18:32:30 +02:00
Michael Eischer
c9191ea72c
forget: cleanup verbose output on snapshot deletion error
2024-04-14 14:17:40 +02:00
Michael Eischer
09587e6c08
repository: duplicate a few blobs in prune tests
2024-04-14 13:57:19 +02:00
Michael Eischer
defd7ae729
prune/repair index: reset in-memory index after command
...
The current in-memory index becomes stale after prune or repair index
have run. Thus, just drop the in-memory index altogether once these
commands have finished.
2024-04-14 13:46:24 +02:00
Michael Eischer
038586dc9d
repository: add minimal test for prune
2024-04-14 13:45:17 +02:00
Michael Eischer
d8622c86eb
prune: clean up internal interface
2024-04-14 13:45:15 +02:00
Michael Eischer
8d507c1372
repository: add basic test for RepairIndex
2024-04-14 13:45:15 +02:00
Michael Eischer
310db03c0e
repair index: improve log output if index cannot be deleted
...
The operation will always fail with an error if an index cannot be
deleted. Thus, this change is purely cosmetic.
2024-04-14 13:45:13 +02:00
Michael Eischer
7d1b9cde34
repository: use normal Init method in tests
2024-04-14 13:45:11 +02:00
Michael Eischer
b25fc2c89d
repository: remove redundant flushes from tests
2024-04-14 13:45:10 +02:00
Michael Eischer
c65459cd8a
repository: speed up tests
2024-04-14 13:45:10 +02:00
Michael Eischer
eda9f7beb4
ui/progress: add helper to print messages during tests
2024-04-14 13:45:08 +02:00
Michael Eischer
35277b7797
backend/mem: cleanup not found error message
2024-04-14 13:45:06 +02:00
Michael Eischer
7ba5e95a82
check: allow tests to only verify pack&index integrity
2024-04-14 13:45:04 +02:00
Michael Eischer
4c9a10ca37
repair packs: deduplicate index rebuild
2024-04-14 13:45:02 +02:00
Michael Eischer
85e4021619
prune: move additional option checks to repository
2024-04-14 13:44:58 +02:00
Michael Eischer
55d56db31b
Merge pull request #4743 from MichaelEischer/deprecate-s3legacy-layout
...
Deprecate s3legacy layout
2024-04-11 22:09:34 +02:00
Michael Eischer
fc3b548625
prune: move logic into repository package
2024-04-10 21:30:52 +02:00
Michael Eischer
df9d4b455d
prune: prepare for moving code to repository package
2024-04-10 21:30:52 +02:00
Michael Eischer
866ddf5698
repair index: refactor code into repository package
2024-04-10 21:30:52 +02:00
Michael Eischer
32a234b67e
prune/forget/repair index: convert output to use progress.Printer
2024-04-10 21:30:52 +02:00
Michael Eischer
739d11c2eb
forget: replace usage of DeleteFilesChecked
...
This simplifies refactoring prune into the repository package.
2024-04-10 21:30:52 +02:00
Michael Eischer
591b421c4a
Deprecate s3legacy layout
2024-04-10 21:27:56 +02:00
Michael Eischer
8efc3a8b7d
Merge pull request #4668 from MichaelEischer/backup-xattr-parent-enoperm
...
backup: Ignore xattr.list permission error for parent directories
2024-04-10 21:25:28 +02:00
Michael Eischer
bf054c09d2
backup: Ignore xattr.list permission error for parent directories
...
On FreeBSD, limited users may not be able to even list xattrs for the
parent directories above the snapshot source paths. As this can cause
the backup to fail, just ignore those errors.
2024-04-10 20:46:15 +02:00
Michael Eischer
0747cf5319
cache: ignore ErrNotExist during cleanup of old files
...
Two restic processes running concurrently can try to remove the same
files from the cache. This could cause one process to fail with an error
if the other one has already remove a file that the current process also
tries to delete.
2024-04-10 19:25:51 +02:00
Michael Eischer
6091029fd6
Merge pull request #4756 from mgeisler/patch-1
...
doc: fix typo in 047_tuning_backup_parameters.rst
2024-04-07 19:24:09 +00:00
Martin Geisler
09d2183351
doc: fix typo in 047_tuning_backup_parameters.rst
2024-04-07 18:05:53 +02:00
Michael Eischer
a4b7ebecfc
Merge pull request #4750 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.40.0
...
build(deps): bump cloud.google.com/go/storage from 1.39.0 to 1.40.0
2024-04-03 20:10:41 +00:00
dependabot[bot]
ba136b31b8
build(deps): bump cloud.google.com/go/storage from 1.39.0 to 1.40.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.39.0 to 1.40.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.39.0...spanner/v1.40.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-04-03 20:03:22 +00:00
Michael Eischer
f328111a6e
Merge pull request #4751 from restic/dependabot/go_modules/golang.org/x/net-0.23.0
...
build(deps): bump golang.org/x/net from 0.21.0 to 0.23.0
2024-04-03 19:52:29 +00:00
Michael Eischer
9fb017e67a
Merge pull request #4745 from MichaelEischer/full-id-key-list
...
key list: include full key id in JSON output
2024-04-03 21:50:02 +02:00
Michael Eischer
49f98f25fc
Merge pull request #4742 from MichaelEischer/consistent-rtest-import
...
Use consistent alias for interal/test package
2024-04-03 21:47:32 +02:00
dependabot[bot]
96c602a6de
build(deps): bump golang.org/x/net from 0.21.0 to 0.23.0
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.21.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.21.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-04-03 19:41:38 +00:00
Michael Eischer
1d0a20dd58
Merge pull request #4748 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.10.0
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.9.2 to 1.10.0
2024-04-03 19:34:47 +00:00
Michael Eischer
6cca1d5705
Merge pull request #4655 from ae-govau/unixsocket
...
Enhancement: option to send HTTP over unix socket
2024-04-03 19:29:21 +00:00
dependabot[bot]
f8a72ac2a3
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.9.2 to 1.10.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.9.2...sdk/azcore/v1.10.0 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-04-01 02:32:19 +00:00
Michael Eischer
5145c8f9c0
key list: include full key id in JSON output
2024-03-31 12:25:20 +02:00
Michael Eischer
831fc4413d
Merge pull request #4737 from stephan0307/3117
...
json output forget command: added id's in snapshots within reasons object
2024-03-29 13:11:24 +00:00
Stephan Paul
df07814ec2
forget json output: added id's in snapshots within reasons object
...
In order to evaluate the keep reasons for snapshots, there should be also the id's
to compare it with snapshots within the keep object. (See also Issue #3117 )
In order to avoid output parameters also changed function addJSONSnapshots to asJSONSnapshots
2024-03-29 11:30:00 +01:00
Michael Eischer
ec2b79834a
use consistent alias for interal/test package
2024-03-29 00:24:03 +01:00
Michael Eischer
510f6f06b0
Merge pull request #4709 from MichaelEischer/refactor-locking
...
Refactor locking into repository package
2024-03-28 23:53:09 +01:00
Michael Eischer
07eb6c315b
add changelog for locking refactor
2024-03-28 23:46:58 +01:00
Michael Eischer
5e98f1e2eb
repository: fix test setup race conditions
2024-03-28 23:17:02 +01:00
Michael Eischer
8155dbe711
correctly lock repository in integration tests
2024-03-28 23:17:02 +01:00
Michael Eischer
d18726cd70
ls: add missing read lock
...
As `ls` reads data from the repository, it must acquire a read lock
unless `--no-lock` was specified. The old behavior is equivalent to `ls
--no-lock`.
2024-03-28 23:17:02 +01:00
Michael Eischer
dc441c57a7
repository: unify repository initialization in tests
...
Tests should use a helper from internal/repository/testing.go to
construct a Repository object.
2024-03-28 23:17:02 +01:00
Michael Eischer
3ba1fa3cee
repository: remove a few global variables
2024-03-28 23:17:02 +01:00
Michael Eischer
044e8bf821
repository: parallelize lock tests
2024-03-28 23:17:02 +01:00
Michael Eischer
e8df50fa3c
repository: remove global list of locks
2024-03-28 22:46:33 +01:00
Michael Eischer
cbb5f89252
lock: move code to repository package
2024-03-28 22:46:33 +01:00
Michael Eischer
118a69a84b
lock: replace lockRepo(Exclusive) with openWith(Read/Write/Exclusive)Lock
...
The new functions much better convey the intent behind the lock
request. This allows cleanly integrating noLock (for read) and dryRun
(write/exclusive) handling.
There are only minor changes to existing behavior with two exceptions:
- `tag` no longer accepts the `--no-lock` flag. As it replaces files in
the repository, this always requires an exclusive lock.
- `debug examine` now returns an error if both `--extract-pack` and
`--no-lock` are given.
2024-03-28 22:46:33 +01:00
Michael Eischer
7f9ad1c3db
Merge pull request #4705 from MichaelEischer/snapshot-statistics
...
Store snapshot statistics & print snapshot size
2024-03-28 22:41:45 +01:00
Michael Eischer
71b6284155
Merge pull request #4006 from MichaelEischer/deviceID-only-for-hardlinks
...
archiver: only store deviceID for hardlinks
2024-03-28 22:33:28 +01:00
Michael Eischer
cf81f8ced6
stats: only check for hardlinks for files with more than one link
2024-03-28 21:29:27 +01:00
Michael Eischer
21cf38fe96
add changelog for deviceID only for hardlinks
2024-03-28 19:32:50 +01:00
Michael Eischer
d705741571
backup: test that deviceID is only stored for hardlinks
2024-03-28 19:32:50 +01:00
Michael Eischer
a9b3d86c4f
features: remove example feature
2024-03-28 19:12:07 +01:00
Michael Eischer
a26d6ffa72
archiver: move deviceID handling behind feature flag
2024-03-28 19:12:07 +01:00
Michael Eischer
2ba21fe72b
archiver: only store deviceID for hardlinks
...
The deviceID can change e.g. when backing up from filesystem snapshot.
It is only used for hardlink detection. Thus there it is not necessary
to store it for everything else.
2024-03-28 19:12:07 +01:00
Michael Eischer
870904d3ae
Merge pull request #4731 from facutuesca/powershell-completion-doc
...
doc: Add instructions to configure PowerShell completions
2024-03-28 17:46:55 +00:00
Facundo Tuesca
15555c9898
doc: Use consistent case for PROFILE env variable in PowerShell
2024-03-28 18:35:18 +01:00
Michael Eischer
63a2350c9e
Merge pull request #4741 from MichaelEischer/ci-upgrade-golangci-lint
...
CI: Update golangci-lint to version 1.57.1
2024-03-28 18:27:05 +01:00
Michael Eischer
8876e3025b
Merge pull request #4724 from MichaelEischer/disable-old-index
...
Deprecated legacy index format
2024-03-28 18:24:47 +01:00
Michael Eischer
4f4979f4e8
Merge pull request #4740 from MichaelEischer/ci-upgrade-docker-actions
...
CI: update docker actions
2024-03-28 18:16:38 +01:00
Michael Eischer
1497525e15
CI: Update golangci-lint to version 1.57.1
2024-03-28 18:15:50 +01:00
Michael Eischer
a8face3a25
Merge pull request #4739 from MichaelEischer/ci-allow-annotations
...
CI: Allow golangci-lint to annotate PRs
2024-03-28 18:02:16 +01:00
Michael Eischer
aee6d311f1
CI: update docker actions
2024-03-28 18:01:48 +01:00
Michael Eischer
6ac7519188
add changelog for rest unix socket support
2024-03-28 17:41:41 +01:00
Michael Eischer
add37fcd9f
CI: uses rest-server from master branch until unix sockets are released
2024-03-28 17:41:41 +01:00
Adam Eijdenberg
6e775d3787
Enhancement: option to send HTTP over unix socket
...
add tests for unix socket connection
switch HTTP rest-server test to use any free port
allow rest-server test graceful shutdown opportunity
2024-03-28 17:41:41 +01:00
Michael Eischer
5c4a4b4a30
CI: Allow golangci-lint to annotate PRs
2024-03-28 17:09:59 +01:00
Michael Eischer
d1d773cfcd
Merge pull request #4727 from avoidalone/master
...
fix some typos
2024-03-17 18:52:40 +00:00
Facundo Tuesca
521713fc94
doc: Add instructions to configure PowerShell completions
2024-03-16 18:54:27 +01:00
avoidalone
ac948fccda
fix some typos
...
Signed-off-by: avoidalone <wuguangdong@outlook.com>
2024-03-11 14:35:12 +08:00
rawtaz
9284f7413a
Merge pull request #4725 from leoheitmannruiz/master
...
Minor README.md cleanups
2024-03-10 12:24:07 +00:00
rawtaz
1287b977b4
Merge pull request #4726 from leoheitmannruiz/patch-1
...
Capitalize Homebrew and minor edit in wording
2024-03-10 12:23:48 +00:00
Leo Heitmann Ruiz
00f762373f
Capitalize Homebrew
2024-03-10 00:20:26 +01:00
Leo Heitmann Ruiz
9f3e1462c0
Minor README.md cleanups
2024-03-09 23:56:16 +01:00
Michael Eischer
69ca12d2eb
check: treat legacy index format as errors
2024-03-09 18:36:33 +01:00
Michael Eischer
98a6817d01
add changelog for legacy index deprecation
2024-03-09 18:35:00 +01:00
Michael Eischer
f8852f0eb6
repair index: fix deletion of legacy indexes
2024-03-09 18:21:22 +01:00
Michael Eischer
1a8bf358f1
index: deprecate legacy index format
2024-03-09 18:21:14 +01:00
Michael Eischer
396a61a992
Merge pull request #4666 from MichaelEischer/feature-flags
...
Implement feature flags
2024-03-09 17:36:29 +01:00
Michael Eischer
a9b64cd7ad
features: print warning for stable/depreacted feature flags
2024-03-09 17:29:52 +01:00
Michael Eischer
fe68d2cafb
add feature flag documentation
2024-03-09 17:29:52 +01:00
Michael Eischer
70839155f2
features: add tests
2024-03-09 17:29:52 +01:00
Michael Eischer
1c77c51a03
features: initialize based on RESTIC_FEATURES environment variable
2024-03-09 17:29:52 +01:00
Michael Eischer
5974a79497
features: add basic feature flag implementation
2024-03-09 17:29:52 +01:00
Michael Eischer
0589da60b3
Merge pull request #4717 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.39.0
...
build(deps): bump cloud.google.com/go/storage from 1.37.0 to 1.39.0
2024-03-07 14:10:24 +00:00
dependabot[bot]
608116817b
build(deps): bump cloud.google.com/go/storage from 1.37.0 to 1.39.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.37.0 to 1.39.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.37.0...spanner/v1.39.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-07 13:59:23 +00:00
Michael Eischer
f742da8b2f
Merge pull request #4722 from konidev20/upgrade-docker-golang-base-image-to-1-22
...
docker: update the base image to golang:1.22-alpine
2024-03-07 13:44:48 +00:00
Michael Eischer
af1684743f
Merge pull request #4716 from restic/dependabot/go_modules/github.com/klauspost/compress-1.17.7
...
build(deps): bump github.com/klauspost/compress from 1.17.6 to 1.17.7
2024-03-07 13:41:46 +00:00
Michael Eischer
5b9de4d8b7
Merge pull request #4715 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob-1.3.1
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 1.2.1 to 1.3.1
2024-03-07 13:40:44 +00:00
Michael Eischer
fceef67abe
Merge pull request #4718 from restic/dependabot/go_modules/github.com/spf13/cobra-1.8.0
...
build(deps): bump github.com/spf13/cobra from 1.7.0 to 1.8.0
2024-03-07 13:37:17 +00:00
Michael Eischer
8506cae710
Merge pull request #4714 from restic/dependabot/github_actions/golangci/golangci-lint-action-4
...
build(deps): bump golangci/golangci-lint-action from 3 to 4
2024-03-07 13:34:21 +00:00
Michael Eischer
87d47ef189
Merge pull request #4713 from restic/dependabot/github_actions/docker/login-action-5139682d94efc37792e6b54386b5b470a68a4737
...
build(deps): bump docker/login-action from 3d58c274f17dffee475a5520cbe67f0a882c4dbb to 5139682d94efc37792e6b54386b5b470a68a4737
2024-03-07 13:33:49 +00:00
Michael Eischer
55abf25ea8
Merge pull request #4719 from restic/dependabot/go_modules/golang.org/x/oauth2-0.17.0
...
build(deps): bump golang.org/x/oauth2 from 0.16.0 to 0.17.0
2024-03-07 13:32:36 +00:00
Srigovind Nayak
b48b1fa2c9
docker: update the base image to golang:1.22-alpine
2024-03-03 16:18:13 +05:30
dependabot[bot]
8e7f29ae28
build(deps): bump golang.org/x/oauth2 from 0.16.0 to 0.17.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.16.0 to 0.17.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.16.0...v0.17.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:25:47 +00:00
dependabot[bot]
79e8ddac3f
build(deps): bump github.com/spf13/cobra from 1.7.0 to 1.8.0
...
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra ) from 1.7.0 to 1.8.0.
- [Release notes](https://github.com/spf13/cobra/releases )
- [Commits](https://github.com/spf13/cobra/compare/v1.7.0...v1.8.0 )
---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:25:41 +00:00
dependabot[bot]
b5a9b5d0bc
build(deps): bump github.com/klauspost/compress from 1.17.6 to 1.17.7
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.17.6 to 1.17.7.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.17.6...v1.17.7 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:25:00 +00:00
dependabot[bot]
f185c80cf0
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go ) from 1.2.1 to 1.3.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azidentity/v1.2.1...sdk/azcore/v1.3.1 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:24:46 +00:00
dependabot[bot]
70c8aaa303
build(deps): bump golangci/golangci-lint-action from 3 to 4
...
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action ) from 3 to 4.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases )
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3...v4 )
---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:17:27 +00:00
dependabot[bot]
e1a588b75c
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from 3d58c274f17dffee475a5520cbe67f0a882c4dbb to 5139682d94efc37792e6b54386b5b470a68a4737.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](3d58c274f1...5139682d94
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:17:20 +00:00
Michael Eischer
e71660cd1e
backup: rename data_added_in_repo statistic to data_added_packed
2024-02-25 20:40:52 +01:00
Aneesh Nireshwalia
062d408987
Clean up SecurityDescriptor helper
2024-02-24 14:23:04 -07:00
Aneesh Nireshwalia
5764300022
Add changelog and fix lint error
2024-02-24 13:47:49 -07:00
Aneesh Nireshwalia
c0a1b9ada5
Update docs for security descriptors
2024-02-24 13:28:18 -07:00
Aneesh Nireshwalia
90916f53de
Add test cases for security descriptors
2024-02-24 13:27:01 -07:00
Aneesh Nireshwalia
70cf8e3788
Add support for backup/restore of security descriptors
2024-02-24 13:25:28 -07:00
Aneesh Nireshwalia
e3e59fef24
Fix CombineErrors and fillExtendedAttr error handling
2024-02-24 13:22:34 -07:00
Aneesh Nireshwalia
09ce1b4e58
Create helper for SecurityDescriptor related functions
2024-02-24 13:16:25 -07:00
Michael Eischer
6a13e451b1
document snapshot statistics
2024-02-23 22:32:04 +01:00
Michael Eischer
a8f5684f68
archiver: test statistics in snapshot
2024-02-23 22:05:15 +01:00
Michael Eischer
681395955e
archiver: test backup summary calculation
2024-02-23 21:46:39 +01:00
Michael Eischer
b6520038fd
snapshots: Print snapshot size stored in snapshots
2024-02-23 20:27:13 +01:00
Michael Eischer
38f91d3b5e
backup: store statistics in snapshot
2024-02-23 20:27:13 +01:00
Michael Eischer
86897314d5
backup: expose data_added_in_repo in JSON output
...
The value describes how much data was added after compression.
Previously, it was only available in the text output.
2024-02-23 20:27:13 +01:00
Michael Eischer
a59f654fa6
archiver: refactor summary collection from ui into the archiver
2024-02-23 20:27:13 +01:00
Michael Eischer
8b1a85711f
archiver: unexport save/saveDir/saveTree methods
2024-02-23 20:24:21 +01:00
Michael Eischer
b953dc8f58
Merge pull request #4611 from zmanda/windows-metadata-support
...
Back up and restore windows metadata like created ts, file attribs like hidden, readonly, encrypted with a common extensible mechanism
2024-02-23 18:16:09 +00:00
Aneesh Nireshwalia
e8211cb64a
Add changelog and update docs for windows attr
2024-02-22 17:59:56 -07:00
Aneesh Nireshwalia
4bbd25a37f
Add tests for generic attribute changes
2024-02-22 17:55:50 -07:00
Aneesh Nireshwalia
d4be734c73
Handle readonly empty files in windows
2024-02-22 17:54:43 -07:00
Aneesh Nireshwalia
eeb1aa5388
Add ability to report warnings to terminal
...
Report warnings to terminal when unrecognized generic attributes are found in the repository.
2024-02-22 17:52:26 -07:00
Aneesh Nireshwalia
0962917974
Support windows metadata using generic attribs
...
Add new generic_attributes attribute in Node.
Use the generic attributes to add support for creation time and file attributes like hidden, readonly, encrypted in windows. Handle permission errors for readonly files in windows.
Handle backup and restore of encrypted attributes using windows system calls.
2024-02-22 17:31:20 -07:00
Aneesh Nireshwalia
62a8a599f1
Add optional messages for Equals helper
2024-02-22 16:58:12 -07:00
Aneesh Nireshwalia
94de87d4b7
Add CombineErrors helper function
2024-02-22 16:57:00 -07:00
Michael Eischer
c6311c1e32
Merge pull request #4703 from ferringb/master
...
Catch SIGTERM, run cleanup
2024-02-22 21:06:29 +00:00
Michael Eischer
0a65a0f94f
update comment
2024-02-22 22:00:42 +01:00
Brian Harring
b41107dcaf
Add changelog for SIGTERM bugfix.
...
Signed-off-by: Brian Harring <ferringb@gmail.com>
2024-02-19 11:31:48 +01:00
Brian Harring
30e979d252
Catch SIGTERM, run cleanup
...
The previous code only ran cleanup (lock release for example) on SIGINT. For
anyone running restic in a container, the signal is going to be SIGTERM which
means containerized execution would leave locks behind.
While this could be addressed via interposing dumb-init to translate the signal,
a `kill` invocation is going to default to SIGTERM, so the same problem exists
for non container users.
Signed-off-by: Brian Harring <ferringb@gmail.com>
2024-02-19 11:12:15 +01:00
Michael Eischer
cfbeb2cde5
Merge pull request #4701 from MichaelEischer/better-streampack-errors
...
repository: Improve StreamPack error messages
2024-02-18 16:57:56 +01:00
Michael Eischer
80754dbf0c
Merge pull request #4664 from MichaelEischer/ls-unified-json-output
...
ls: include standard `message_type` field in output
2024-02-18 15:47:41 +00:00
Michael Eischer
4c3218ef9f
repository: include packID in StreamPack for decrypt/decompress errors
2024-02-17 19:38:01 +01:00
Michael Eischer
18b0bbbf42
repository: use fmt.Errorf in StreamPacks
2024-02-17 19:37:32 +01:00
Michael Eischer
6fbb470835
Merge pull request #4665 from MichaelEischer/check-repair-packs
...
check: Suggest usage of `repair packs` if pack files are damaged
2024-02-17 15:54:07 +00:00
Michael Eischer
0a36d193d8
add changelog for enhanced repair packs
2024-02-12 21:43:35 +01:00
Michael Eischer
69304cd74f
check: clarify repair pack usage
2024-02-12 21:43:35 +01:00
Michael Eischer
c3b0e6d004
Merge pull request #4700 from MichaelEischer/remove-duplicate-changelog-entries
...
remove changelogs that are already included in restic 0.16.4
2024-02-12 21:40:04 +01:00
Michael Eischer
9e3703ded5
remove changelogs that are already included in restic 0.16.4
2024-02-12 20:39:31 +01:00
Michael Eischer
527a3ff2b2
check: link to troubleshooting guide
2024-02-12 20:25:15 +01:00
Michael Eischer
ed4a4f8748
check: exclude inaccessible files from the repair pack suggestion
2024-02-12 20:25:15 +01:00
Michael Eischer
4073299a7c
check: fix missing error if blob is invalid
2024-02-12 20:20:13 +01:00
Michael Eischer
6397615fbb
check: document that check will show repair pack instructions
2024-02-12 20:20:13 +01:00
Michael Eischer
544fe38786
check: suggest repair pack for all damaged packs
2024-02-12 20:20:13 +01:00
Michael Eischer
772e3416d1
repair pack: drop feature flag
2024-02-12 20:20:12 +01:00
Michael Eischer
22a3cea1b3
checker: wrap all pack errors in ErrPackData
2024-02-12 20:19:32 +01:00
Michael Eischer
19bf2cf52d
Merge pull request #4697 from MichaelEischer/report-blob-errors
...
backup: report files whose chunks failed to upload
2024-02-12 20:18:51 +01:00
Michael Eischer
5b5d506472
backup: report files whose chunks failed to upload
2024-02-11 22:43:26 +01:00
Michael Eischer
dde556e8e8
Merge pull request #4696 from MichaelEischer/fix-exclude-load-error-msg
...
backup: improve error message if exclude file cannot be loaded
2024-02-11 21:35:31 +00:00
Michael Eischer
ee1ff3c1d0
backup: improve error message if exclude file cannot be loaded
2024-02-11 22:26:13 +01:00
Michael Eischer
667a2f5369
Merge pull request #4694 from restic/update-go-versions
...
Update go versions
2024-02-11 17:10:12 +00:00
Michael Eischer
2ab18a92e6
CI: keep tests for Go 1.19
2024-02-10 23:42:34 +01:00
Alexander Neumann
c0514dd8ba
Fix linter errors (except for tests)
2024-02-10 22:58:10 +01:00
Alexander Neumann
a8cda0119c
Upgrade golangci-lint
2024-02-10 22:08:43 +01:00
Alexander Neumann
9720935c56
Update Go version for tests to 1.22
2024-02-10 21:56:01 +01:00
Michael Eischer
68cc327b15
Merge pull request #4692 from 27149chen/dump-to-existing-file
...
feat: dump flag --target should be allowed to write existing file
2024-02-10 17:44:54 +00:00
Michael Eischer
15d6fa1f83
dump: update docs for --target option
2024-02-10 18:39:06 +01:00
lou
80db02fc35
dump flag --target should be allowed to write existing file
...
Signed-off-by: lou <alex1988@outlook.com>
2024-02-10 18:39:06 +01:00
Michael Eischer
6a2b10e2a8
Merge pull request #4685 from konidev20/fix-gh-4676-sub-commands-for-key-management
...
Move key add, list, remove and passwd as separate sub-commands and improve key sub-command documentation
2024-02-08 19:59:16 +00:00
Michael Eischer
e46b21ab80
key: fix integration test for invalid arguments
2024-02-08 20:52:30 +01:00
Michael Eischer
eb389a2d25
Merge pull request #4687 from MichaelEischer/upgrade-zstd-library
...
Upgrade zstd library to latest version
2024-02-08 20:29:13 +01:00
Srigovind Nayak
795d33b3ee
key: move add, list, remove, passwd to sub-commands
...
docs: improve the sub-command docs
changelog: add the unreleased changelog for the key command updates
key: update integration tests
2024-02-06 01:47:43 +05:30
Michael Eischer
0cffdb7493
Merge pull request #4682 from konidev20/feat-add-target-for-dump-command
...
feat: set --target for dump command
2024-02-05 19:16:42 +00:00
Michael Eischer
f5ffa40652
dump: minor cleanups
2024-02-05 20:10:52 +01:00
Srigovind Nayak
175c14b5c9
dump: add --target option
2024-02-05 20:10:52 +01:00
Michael Eischer
bca099ac7f
Upgrade zstd library to latest version
...
The data corruption bug is fixed, thus remove the override.
2024-02-05 19:53:02 +01:00
Michael Eischer
0f09a8870c
Merge pull request #4684 from konidev20/fix-gh-4658-update-azure-documentation
...
Update azure storage account authentication documentation
2024-02-04 20:13:18 +00:00
Srigovind Nayak
5771c4ecfb
docs: update environment variables for `az login` to azure backend
2024-02-05 01:32:43 +05:30
Michael Eischer
b63bfd2257
Merge branch 'patch-release'
2024-02-04 20:21:42 +01:00
Alexander Neumann
0f9fa44de5
Set development version for 0.16.4
2024-02-04 19:50:56 +01:00
Alexander Neumann
3786536dc1
Add version for 0.16.4
2024-02-04 19:50:52 +01:00
Alexander Neumann
811be5984d
Update manpages and auto-completion
2024-02-04 19:50:51 +01:00
Alexander Neumann
b0ead75de5
Generate CHANGELOG.md for 0.16.4
2024-02-04 19:50:34 +01:00
Alexander Neumann
6cd2804bff
Prepare changelog for 0.16.4
2024-02-04 19:50:34 +01:00
Michael Eischer
a72c2b74f3
Apply changelog entry / documentation improvements from review
2024-02-04 19:10:06 +01:00
Michael Eischer
261b1455c7
add documentation for --no-extra-verify option
2024-02-04 19:10:06 +01:00
Michael Eischer
2a0bd2b637
rename `--no-verify-pack` to `--no-extra-verify`
2024-02-04 19:10:05 +01:00
Michael Eischer
4589da7eb9
add data verification changelog entry
2024-02-04 19:09:49 +01:00
Michael Eischer
75e72d826c
pack: verify integrity of pack file header
2024-02-04 19:09:49 +01:00
Michael Eischer
d8916bc3d9
repository: ask users to report corrupted data while saving blobs
2024-02-04 19:09:49 +01:00
Michael Eischer
dc11d012bb
Make --no-verify-pack globally available
...
Verifying all blobs before upload comes with a notable performance
impact. Allow users to skip it if necessary.
2024-02-04 19:09:49 +01:00
Michael Eischer
8ef5425351
repository: test verification of blobs/unpacked data
2024-02-04 19:09:46 +01:00
Michael Eischer
885431ec2b
repository: Allow skipping verification for tests
...
Some tests have to explicitly create pack files with blobs that don't
match their ID. For those blobs the builtin verification of the
repository must be disabled.
2024-02-04 19:08:30 +01:00
Michael Eischer
cb85fb46dd
backup: verify unpacked files before upload
2024-02-04 19:07:48 +01:00
Michael Eischer
2f30c940b2
backup: verify blobs before upload
...
This only covers the blobs themselves, the pack header is not verified
so far. Unpacked files are also not covered by the integrity check.
2024-02-04 19:07:48 +01:00
Michael Eischer
0ea62b5ac6
repository: make repo.Options configurable for test repos
2024-02-04 19:07:46 +01:00
Michael Eischer
29e1caf825
add changelog draft for data corruption on max compression
2024-02-04 19:05:51 +01:00
Michael Eischer
0164f5310d
Downgrade klauspost/compress to fix data corruption at max. compression
2024-02-04 19:05:50 +01:00
Michael Eischer
d5e662315a
Merge pull request #4681 from MichaelEischer/verify-integrity-on-upload
...
backup: verify blobs before upload
2024-02-04 18:04:27 +00:00
Michael Eischer
effe76aaf5
Merge pull request #4679 from MichaelEischer/workaround-compression-bug
...
Downgrade klauspost/compress to fix data corruption at max. compression
2024-02-04 18:03:34 +00:00
Michael Eischer
5957417b1f
Apply changelog entry / documentation improvements from review
2024-02-04 18:55:41 +01:00
Michael Eischer
219d8e3c18
add changelog draft for data corruption on max compression
2024-02-04 18:11:48 +01:00
Michael Eischer
a737fe1e47
add documentation for --no-extra-verify option
2024-02-04 17:11:49 +01:00
Michael Eischer
86b38a0b17
rename `--no-verify-pack` to `--no-extra-verify`
2024-02-04 17:01:05 +01:00
Michael Eischer
7d31180fe6
add data verification changelog entry
2024-02-04 15:48:11 +01:00
Michael Eischer
c32e5e2abb
pack: verify integrity of pack file header
2024-02-04 15:31:42 +01:00
Michael Eischer
c97a271e89
repository: ask users to report corrupted data while saving blobs
2024-02-04 15:31:42 +01:00
Michael Eischer
66e8971659
Make --no-verify-pack globally available
...
Verifying all blobs before upload comes with a notable performance
impact. Allow users to skip it if necessary.
2024-02-04 15:31:42 +01:00
Michael Eischer
193140525c
repository: test verification of blobs/unpacked data
2024-02-04 15:31:42 +01:00
Michael Eischer
96518d7c4a
Merge pull request #4674 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.37.0
...
build(deps): bump cloud.google.com/go/storage from 1.34.0 to 1.37.0
2024-02-03 17:23:49 +00:00
Michael Eischer
2dbb18128c
repository: Allow skipping verification for tests
...
Some tests have to explicitly create pack files with blobs that don't
match their ID. For those blobs the builtin verification of the
repository must be disabled.
2024-02-03 18:22:47 +01:00
Michael Eischer
30a84e9003
backup: verify unpacked files before upload
2024-02-03 18:22:47 +01:00
Michael Eischer
c01a0c6da7
backup: verify blobs before upload
...
This only covers the blobs themselves, the pack header is not verified
so far. Unpacked files are also not covered by the integrity check.
2024-02-03 18:22:47 +01:00
Michael Eischer
16e3f79e8b
repository: make repo.Options configurable for test repos
2024-02-03 18:22:47 +01:00
Michael Eischer
bb92b487f7
repository: fix repack test
2024-02-03 18:22:47 +01:00
dependabot[bot]
cf7cad11de
build(deps): bump cloud.google.com/go/storage from 1.34.0 to 1.37.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.34.0 to 1.37.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.34.0...spanner/v1.37.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-02-02 20:40:57 +00:00
Michael Eischer
370d9c31f4
Merge pull request #4671 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob-1.2.1
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 1.2.0 to 1.2.1
2024-02-02 20:31:16 +00:00
Michael Eischer
6581133e85
Merge pull request #4675 from restic/dependabot/go_modules/golang.org/x/oauth2-0.16.0
...
build(deps): bump golang.org/x/oauth2 from 0.15.0 to 0.16.0
2024-02-02 20:29:39 +00:00
Michael Eischer
207a4a5e8e
Downgrade klauspost/compress to fix data corruption at max. compression
2024-02-02 20:10:29 +01:00
Michael Eischer
cbf9cd4a7f
Merge pull request #4670 from joram-berger/patch-1
...
Link to Go Match syntax directly in 040_backup.rst
2024-02-02 18:40:13 +00:00
dependabot[bot]
552f01662b
build(deps): bump golang.org/x/oauth2 from 0.15.0 to 0.16.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.15.0 to 0.16.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.15.0...v0.16.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-02-01 01:53:12 +00:00
dependabot[bot]
7f5ea511bc
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go ) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/v1.2...sdk/azidentity/v1.2.1 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-02-01 01:51:32 +00:00
Joram Berger
b07afa9b02
Link to Go Match syntax directly in 040_backup.rst
...
The docs linked to filepath.Glob (which is used) but the syntax is described in the documentation of filepath.Match. So it makes sense to link that directly.
2024-01-31 23:40:21 +01:00
Michael Eischer
8b08b522c9
Merge pull request #4669 from MichaelEischer/fix-rewrite-typo
...
rewrite: fix typo in dry-run output
2024-01-31 22:08:12 +01:00
Michael Eischer
eaf9659efc
Merge pull request #4657 from numerigraphe/fix-key-add-id
...
Properly report the ID of newly added keys
2024-01-31 20:59:41 +00:00
Michael Eischer
ba136ff60c
rewrite: fix typo in dry-run output
2024-01-31 21:48:37 +01:00
Lionel Sausin
8fbe328371
Properly report the ID of newly added keys
...
Other commands like key list and key remove show the key's ID.
Showing the ID here lets users easily reuse the ID as a key hint for subsequent
commands.
In particular, a key hint is needed when the repository has many keys - otherwise
opening the repository may fail with "Fatal: maximum number of keys reached" even
when a proper password is provided.
Fixes #4656
2024-01-29 10:12:49 +01:00
Michael Eischer
4273e06a43
Merge pull request #4662 from MichaelEischer/clarify-backup-ignore-inode
...
backup: clarify that --ignore-inode also ignores ctime
2024-01-27 18:25:11 +01:00
Michael Eischer
248c144f72
Merge pull request #4663 from MichaelEischer/key-subcommand-bugfix
...
key: return an error if subcommand is unknown
2024-01-27 18:24:50 +01:00
Michael Eischer
5dca8a70d5
ls: include standard `message_type` field in output
2024-01-27 15:48:24 +01:00
Michael Eischer
765729d009
key: return an error if subcommand is unknown
2024-01-27 15:33:49 +01:00
Michael Eischer
a09d51d96c
backup: clarify that --ignore-inode also ignores ctime
2024-01-27 13:42:29 +01:00
Michael Eischer
e44e4b00a6
Merge pull request #4550 from ndecker/ls-ncdu
...
Ls ncdu
2024-01-27 12:27:35 +00:00
Michael Eischer
10e71af759
describe ls command in docs
2024-01-27 13:22:00 +01:00
Michael Eischer
c90f24a06c
Merge pull request #4641 from MichaelEischer/reduce-restic-repository-usage
...
Misc cleanups
2024-01-27 13:18:20 +01:00
Michael Eischer
d4ed7c8858
walker: add tests for leaveDir
2024-01-27 13:17:33 +01:00
Michael Eischer
2c80cfa4a5
walker: fix missing leaveDir if directory is partially skipped
2024-01-27 13:17:33 +01:00
Michael Eischer
261737abc8
ls: only allow either --json or --ncdu
2024-01-27 13:17:33 +01:00
Michael Eischer
a2f2f8fb4c
fix linter warning
2024-01-27 13:17:33 +01:00
Michael Eischer
4bae54d040
ls: test ncdu output format
2024-01-27 13:17:33 +01:00
Michael Eischer
509b339d54
ls: correctly handle setuid/setgit/sticky bit in ncdu output
2024-01-27 13:17:33 +01:00
Michael Eischer
a2fe337610
ls: unify printer implementations
2024-01-27 13:17:33 +01:00
Michael Eischer
1b008c92d3
ls: rework ncdu output to use walker.LeaveDir
2024-01-27 13:17:33 +01:00
Michael Eischer
9ecbda059c
walker: add callback to inform about leaving a directory
2024-01-27 13:17:32 +01:00
Nils Decker
b2703a4089
add changelog for ls --ncdu
2024-01-27 13:17:15 +01:00
Nils Decker
a9310948cf
command ls: add option for ncdu output
...
NCDU (NCurses Disk Usage) is a tool to analyse disk usage of directories.
It has an option to save a directory tree and analyse it later.
This patch adds an output option to the ls command.
A snapshot can be seen with
`restic ls latest --ncdu | ncdu -f -`
- https://dev.yorhel.nl/ncdu
2024-01-27 13:06:52 +01:00
Michael Eischer
246559e654
check: cleanup s3 legacy detection
2024-01-27 13:02:04 +01:00
Michael Eischer
1dfd854769
lock: switch to repository.List
2024-01-27 13:02:04 +01:00
Michael Eischer
bfb56b78e1
replace some usages of restic.Repository with more specific interface
...
This should eventually make it easier to test the code.
2024-01-27 13:02:02 +01:00
Michael Eischer
3424088274
Merge pull request #4644 from MichaelEischer/refactor-repair-packs
...
Refactor and test `repair packs`
2024-01-27 13:00:51 +01:00
Michael Eischer
724ec179e3
Merge pull request #4648 from MichaelEischer/repository-removekey
...
repository: Introduce RemoveKey function
2024-01-27 13:00:42 +01:00
Michael Eischer
f0e1ad2285
fix linter warning
2024-01-27 12:51:45 +01:00
Michael Eischer
fd579421dd
repository: deduplicate test
2024-01-27 12:51:45 +01:00
Michael Eischer
42c9318b9c
repair pack: add tests
2024-01-27 12:51:45 +01:00
Michael Eischer
764b0bacd6
repair pack: add support for truncated files
2024-01-27 12:51:45 +01:00
Michael Eischer
7c351bc53c
repair pack: reenable auto index updates
...
The method is not available on the restic.Repository interface that is
used for testing. Drop the call as a small amount of additional index
writes is not a problem.
2024-01-27 12:51:45 +01:00
Michael Eischer
feeab84204
repair pack: extract the repair logic into the repository package
...
Currently, the cmd/restic package contains a significant amount of code
that modifies repository internals. This code should in the mid-term
move into the repository package.
2024-01-27 12:51:45 +01:00
Michael Eischer
d7a50fe739
properly show termstatus progress bar if visible less than one frame
...
If a progress bar using termstatus was only visible for less than one
frame, then its output could be lost.
2024-01-27 12:51:40 +01:00
Michael Eischer
6b65a495b1
backup/restore: fix termstatus initialization
...
The termstatus must only be canceled once the command has returned.
Otherwise output may be lost when the context gets canceled.
2024-01-27 12:51:08 +01:00
Michael Eischer
d26d2d41f8
backup/restore: extract termstatus initialization
2024-01-27 12:51:08 +01:00
Michael Eischer
cb50832d50
index: let MasterIndex.Save also delete obsolete indexes
2024-01-27 12:51:08 +01:00
Michael Eischer
bedff1ed6d
split deleteFiles into UI and logic parts
2024-01-27 12:51:08 +01:00
Michael Eischer
c13bf0b607
repository: Introduce RemoveKey function
...
This replaces directly removing keys via the backend.
2024-01-27 12:42:58 +01:00
Michael Eischer
25ac1549e7
Merge pull request #4661 from MichaelEischer/clarify-contributing
...
CONTRIBUTING.md: Clarify handling of small bugfixes
2024-01-26 23:04:58 +00:00
Michael Eischer
ae9683336d
CONTRIBUTING.md: Clarify handling of small bugfixes
...
Opening an issue for a small bugfix is usually not useful. It primarily
adds overhead.
2024-01-26 23:51:54 +01:00
Michael Eischer
446167ae80
Merge pull request #4643 from MichaelEischer/remove-redundant-poly1305-mask
...
Remove redundant poly1305 key masking
2024-01-23 19:46:40 +01:00
Michael Eischer
5b36c4eb5f
Merge pull request #4647 from MichaelEischer/reduce-globals
...
Remove all usages of the global command-specific options
2024-01-23 19:46:15 +01:00
Michael Eischer
1419baf67a
Merge pull request #4645 from MichaelEischer/improve-lock-checking
...
lock: checkForOtherLocks processes each lock at most once
2024-01-23 19:46:00 +01:00
Michael Eischer
66103aea3d
Remove all usages of the global command-specific options
...
Now, every command uses an options struct, which is passed to the run*
function by the command.RunE method.
2024-01-23 19:21:39 +01:00
Michael Eischer
79f2939eb9
Merge pull request #4654 from adrian5/docfix
...
docs: fix formatting
2024-01-23 18:08:59 +00:00
Michael Eischer
0e2ee06803
Merge pull request #4650 from MichaelEischer/improve-stdin-from-command-description
...
backup: Improve help text for `--stdin-from-command`
2024-01-23 19:22:15 +01:00
Michael Eischer
2927982256
Merge pull request #4649 from MichaelEischer/simplify-termstatus-shutdown
...
ui/termstatus: simplify cleaning up on termination
2024-01-23 19:16:21 +01:00
Michael Eischer
6cc2bec5dd
apply suggestion from review
2024-01-23 19:09:04 +01:00
Michael Eischer
18806944f6
doc: remove blockquotes from unordered lists
2024-01-23 19:03:54 +01:00
adrian5
609f84e095
docs: fix formatting
2024-01-22 21:12:12 +01:00
Michael Eischer
767c2539a0
backup: Improve help text for `--stdin-from-command`
2024-01-21 22:06:54 +01:00
Michael Eischer
6bdca13603
ui/termstatus: simplify cleaning up on termination
...
`writeStatus` also cleans no longer used status lines.
The old code actually cleaned one line too much. However, as that line
was never used it makes no difference.
2024-01-21 21:27:27 +01:00
Michael Eischer
f1f34eb3e5
lock: checkForOtherLocks processes each lock at most once
...
If a lock could not be loaded, then restic would check all lock files
again. These repeated checks are not useful as the status of a lock file
cannot change unless its ID changes too. Thus, skip already check lock
files on retries.
2024-01-20 22:40:12 +01:00
Michael Eischer
fee83e1c09
Remove redundant poly1305 key masking
...
The implementation in crypto/poly1305 already performs the exact same
masking.
2024-01-20 12:36:59 +01:00
Michael Eischer
6696195f38
Merge pull request #4584 from elkemper/fix-stop-archiving-metadata
...
S3: Don't archive metadata files on S3 Glacier
2024-01-20 10:30:37 +00:00
Michael Eischer
a763a5c67d
s3: minor cleanups for archive storage class handling
2024-01-20 11:25:28 +01:00
Vladislav Belous
8ca58b487c
S3: do not set storage class for metadata when using archive storage
2024-01-20 11:04:15 +01:00
Michael Eischer
62111f4379
Merge pull request #4625 from MichaelEischer/refactor-streampacks
...
Refactor repository.StreamPacks
2024-01-19 21:48:37 +01:00
Michael Eischer
2c310a526e
repository: Replace StreamPack function with LoadBlobsFromPack method
...
LoadBlobsFromPack is now part of the repository struct. This ensures
that users of that method don't have to deal will internals of the
repository implementation.
The filerestorer tests now also contain far fewer pack file
implementation details.
2024-01-19 21:40:43 +01:00
Michael Eischer
6b7b5c89e9
repository: prepare StreamPack refactor
2024-01-19 21:40:43 +01:00
Michael Eischer
22d0c3f8dc
check: Use PackBlobIterator instead of StreamPack
...
To only stream the content of a pack file once, check used StreamPack
with a custom pack load function. This combination was always brittle
and complicates using StreamPack everywhere else. Now that StreamPack
internally uses PackBlobIterator use that primitive instead, which is a
much better fit for what the check command requires.
2024-01-19 21:40:36 +01:00
Michael Eischer
fb422497af
repository: split StreamPack implementation
...
Move the actual decoding of the pack data into a separate iterator.
2024-01-19 21:39:55 +01:00
Michael Eischer
54c5c72e5a
Merge pull request #4616 from MichaelEischer/fix-rest-connection-close
...
rest: fix and cleanup closing of http response body
2024-01-19 21:31:35 +01:00
Michael Eischer
5f49eec655
Merge pull request #4615 from MichaelEischer/fix-find-empty-dirs
...
walker: Remove ignoreTrees functionality
2024-01-19 21:25:40 +01:00
Michael Eischer
ec13105093
Merge pull request #4623 from MichaelEischer/docs-verify-release-binaries
...
Add documentation for the verify-release-binaries.sh script
2024-01-19 21:17:38 +01:00
Michael Eischer
bd883caae1
CI: enable bodyclose linter
2024-01-19 21:17:18 +01:00
Michael Eischer
b1a8fd1d03
rest: fix and cleanup closing of http response body
...
If client.Do returns an error, then there's no body that has to be
closed. For requests for which we are not interested in the response
body, immediately drain and close the body to make sure it isn't
forgotten later on.
This change in particular adds the missing `Close()` call for the
`List()` command.
2024-01-19 21:17:17 +01:00
Michael Eischer
fdcbb53017
walker: test skipping for root node
2024-01-19 21:16:06 +01:00
Michael Eischer
0b39940fdb
walker: Remove ignoreTrees functionality
...
It was only used in two places:
- stats: apparently as a minor performance optimization, which is
unlikely to be important
- find: filtered directories would be ignored. However, this
optimization missed that it is possible that two directories have the
exact same content. Such directories would be incorrectly ignored too.
Example:
```
mkdir test test/a test/b
restic backup test
restic find latest test/b
-> incorrectly does not return anything
```
Thus, remove the functionality as it's apparently too complex to use
correctly.
2024-01-19 21:16:06 +01:00
Michael Eischer
147b0e54cb
Merge pull request #4639 from northben/patch-1
...
Update Backblaze documentation
2024-01-19 20:08:48 +00:00
Ben Northway
5413877d33
Update Backblaze documentation
...
clarify documentation regarding B2 bucket lifecycle settings. The default lifecycle setting is probably fine for most users now; a custom policy is not necessary.
2024-01-18 16:41:01 -06:00
Michael Eischer
03e06d0797
Merge branch 'patch-release'
2024-01-14 21:38:17 +01:00
Alexander Neumann
0ec9383ba2
Set development version for 0.16.3
2024-01-14 20:21:45 +01:00
Alexander Neumann
abca112404
Add version for 0.16.3
2024-01-14 20:21:45 +01:00
Alexander Neumann
b70b94507a
Generate CHANGELOG.md for 0.16.3
2024-01-14 20:21:19 +01:00
Alexander Neumann
d987582594
Prepare changelog for 0.16.3
2024-01-14 20:21:19 +01:00
Leo R. Lundgren
ef2e473b99
doc: Polish changelogs
2024-01-10 00:19:07 +01:00
Michael Eischer
7b2de84763
Merge pull request #4618 from MichaelEischer/workaround-rclone-list-errors
...
rclone: Workaround for incorrect "not found" errors while listing files
2024-01-09 18:28:31 +01:00
Michael Eischer
e4bbde7036
rclone: Workaround for incorrect "not found" errors while listing files
...
rclone returns a "not found" error if an internal error occurs while
listing a folder. Ignoring this error lets restic erroneously think that
there are no files, which can cause `prune` to wipe the whole
repository.
2024-01-09 18:28:17 +01:00
Michael Eischer
ec0fb46f6c
add changelog for reliable restores
2024-01-09 18:27:48 +01:00
Michael Eischer
103beb96bc
restore: separately restore blobs that are frequently referenced
...
Writing these blobs to their files can take a long time and consequently
cause the backend connection to time out. Avoid that by retrieving these
blobs separately.
2024-01-09 18:27:48 +01:00
Michael Eischer
f0f89d7f27
restore: split error reporting from downloadPack
2024-01-09 18:27:48 +01:00
Michael Eischer
cf352ccafb
restore: cleanup downloadPack
2024-01-09 18:27:48 +01:00
Michael Eischer
b856e9489a
restore: split downloadPack into smaller methods
2024-01-09 18:27:48 +01:00
Michael Eischer
c31e9418ba
Merge pull request #4626 from MichaelEischer/reliable-large-restores
...
Improve reliability of large restores
2024-01-09 18:23:09 +01:00
Michael Eischer
2e8de9edfd
rclone: Workaround for incorrect "not found" errors while listing files
...
rclone returns a "not found" error if an internal error occurs while
listing a folder. Ignoring this error lets restic erroneously think that
there are no files, which can cause `prune` to wipe the whole
repository.
2024-01-09 18:20:16 +01:00
Michael Eischer
ce7db90e08
sync CI and go dependencies with master branch
2024-01-08 21:33:05 +01:00
Michael Eischer
620518aec6
add changelog for better restore error reporting
2024-01-08 21:33:05 +01:00
Michael Eischer
f2fafbffaa
restore: only report errors for blobs that actually failed to load
...
Previously, errors would be reported for all blobs of a packfile that
failed to stream. Now, only the not yet processed blobs are reported.
2024-01-08 21:33:05 +01:00
Michael Eischer
7a3a884874
repository: test that StreamPack only delivers blobs once
2024-01-08 21:33:05 +01:00
Michael Eischer
772a907533
repository: StreamPack delivers blobs at most once
...
If an error occurred while streaming a pack file, this could result in
passing some of the blobs multiple times to the callback function. This
significantly complicates using StreamPack correctly and is unnecessary.
Retries do not change the content of a blob and thus only deliver the
same result over and over again.
2024-01-08 21:33:05 +01:00
Michael Eischer
a9446c1184
add changelog for irregular files on windows
2024-01-08 21:33:05 +01:00
Michael Eischer
1bab29c336
archiver: Add filepath to error message if it is not included yet
2024-01-08 21:33:05 +01:00
Michael Eischer
e886c3f6b2
archiver: improve error message for irregular files
...
Since Go 1.21, most reparse points are considered as irregular files.
Depending on the underlying driver these can exhibit nearly arbitrary
behavior. When encountering such a file, restic returned an
indecipherable error message: `error: invalid node type ""`.
Add the filepath to the error message and state that the file type is
not supported.
2024-01-08 21:33:05 +01:00
Michael Eischer
c95de54726
restic: cleanup node type determination
...
os.ModeCharDevice is already included in os.ModeType
2024-01-08 21:33:05 +01:00
Michael Eischer
d4b8abd3e2
fix deduplicated files on windows
2024-01-08 21:33:05 +01:00
Joram Berger
948ab3ccaf
Add a note that the oldest snapshot may be kept additionally
...
Documentation enhancement.
2024-01-08 21:33:05 +01:00
Markus Zoppelt
bb0c923298
docs: add pkgx install option
...
PR in pkgx pantry: https://github.com/pkgxdev/pantry/pull/4098
restic pkg:
https://pkgx.dev/pkgs/restic.net/restic/
2024-01-08 21:33:05 +01:00
Michael Eischer
ff0c975443
regenerate changelog
2024-01-08 21:33:05 +01:00
Michael Eischer
7e61e117d6
cleanup changelog whitespace
2024-01-08 21:33:05 +01:00
mmattel
220a28582e
Add a table of contents (TOC) to the changelog template
2024-01-08 21:33:05 +01:00
Giuseppe D'Andrea
f44fd73230
docs: fix typo in working with repos
...
When using the `copy` command, `--from-password-file` and `--from-password-command` flags are used to specify the password of the source repository, not of the destination repository.
2024-01-08 21:33:05 +01:00
Joda Stößer
76bd975e03
docs(scripting): correct stats output comment to be about the correct command
...
not about the snapshots command
2024-01-08 21:33:05 +01:00
Quang-Linh LE
64b7aed362
docs: Mention progress for restore command. This is available after https://github.com/restic/restic/pull/3991
2024-01-08 21:33:05 +01:00
Michael Kuhn
3fa6b2de4a
Fix repository not being printed when using repository file
...
When using `RESTIC_REPOSITORY_FILE` in combination with `restic init`,
the repository is missing in the output:
```
$ restic init
created restic repository 3c872be20f at
[...]
```
This is due to the code using `gopts.Repo`, which is empty in this case.
2024-01-08 21:33:05 +01:00
Michael Eischer
5cd000f4b0
CI: update golangci-lint
...
Necessary to properly support Go 1.21.
2024-01-08 21:33:05 +01:00
Michael Eischer
4ea3796455
add changelog for reliable restores
2024-01-08 21:03:10 +01:00
Michael Eischer
e78be75d1e
restore: separately restore blobs that are frequently referenced
...
Writing these blobs to their files can take a long time and consequently
cause the backend connection to time out. Avoid that by retrieving these
blobs separately.
2024-01-08 21:00:13 +01:00
Michael Eischer
2267910418
restore: split error reporting from downloadPack
2024-01-08 20:57:00 +01:00
Michael Eischer
00d18b7a88
restore: cleanup downloadPack
2024-01-08 20:53:08 +01:00
Michael Eischer
9328f34d43
restore: split downloadPack into smaller methods
2024-01-08 20:52:36 +01:00
Michael Eischer
77434c6e2b
Merge pull request #4474 from ekristen/aws-assume-role
...
Allow AWS Assume Role
2024-01-08 19:07:17 +00:00
Michael Eischer
4248c6c3ca
s3: update documentation
2024-01-07 19:30:11 +01:00
Michael Eischer
e4a7eb09ef
Merge pull request #4624 from MichaelEischer/better-restorer-error-reporting
...
Improver restorer error reporting
2024-01-07 11:20:29 +01:00
Michael Eischer
f8b4e932ef
Merge pull request #4620 from MichaelEischer/improve-irregular-file-handling
...
Improve irregular file handling
2024-01-07 11:12:07 +01:00
Michael Eischer
100872308f
add changelog for better restore error reporting
2024-01-07 11:06:42 +01:00
Michael Eischer
dac3508170
restore: only report errors for blobs that actually failed to load
...
Previously, errors would be reported for all blobs of a packfile that
failed to stream. Now, only the not yet processed blobs are reported.
2024-01-07 10:54:56 +01:00
Michael Eischer
77b1c52673
repository: test that StreamPack only delivers blobs once
2024-01-07 10:54:53 +01:00
Michael Eischer
fe5c337ca2
repository: StreamPack delivers blobs at most once
...
If an error occurred while streaming a pack file, this could result in
passing some of the blobs multiple times to the callback function. This
significantly complicates using StreamPack correctly and is unnecessary.
Retries do not change the content of a blob and thus only deliver the
same result over and over again.
2024-01-07 10:54:49 +01:00
Michael Eischer
3e29f8dddf
add changelog for irregular files on windows
2024-01-07 10:52:12 +01:00
Michael Eischer
76f507c775
Merge pull request #4621 from MichaelEischer/fix-windows-dedup-files
...
Fix backup of deduplicated files on windows
2024-01-07 10:44:50 +01:00
Michael Eischer
6ef23b401b
fix deduplicated files on windows
2024-01-07 10:23:31 +01:00
Michael Eischer
62f99a3b2f
Add documentation for the verify-release-binaries.sh script
2024-01-07 10:16:30 +01:00
Michael Eischer
0360e540af
Merge pull request #4622 from MichaelEischer/fix-outdated-windows-import
...
termstatus: update import path of golang.org/x/term
2024-01-06 23:24:16 +01:00
Michael Eischer
e6dfefba13
termstatus: update import path of golang.org/x/term
2024-01-06 21:59:26 +01:00
Michael Eischer
02bc73f5eb
s3: minor code cleanups
2024-01-06 21:44:53 +01:00
Michael Eischer
20cf4777cb
s3: check for EnvAWS credentials before Static credentials
...
EnvAWS considers more environment variables, including AWS_SESSION_TOKEN
and thus should be checked first.
2024-01-06 21:43:47 +01:00
Erik Kristensen
5ffb536aae
feat: support AWS assume role
2024-01-06 21:19:58 +01:00
Michael Eischer
1604922360
Merge pull request #4527 from adamantike/cmd/copy/prefix-hostname-to-snapshot-paths
...
cmd: Add hostname to snapshot display output
2024-01-06 19:32:14 +00:00
Michael Eischer
c7844530d8
update docs
2024-01-06 20:25:24 +01:00
Michael Eischer
33b7c84a7a
deduplicate string formatting of snapshot metadata
...
This removes the spurious ")" bracket at the end and normalizes the
metadata format used by the `ls` command.
2024-01-06 20:20:51 +01:00
Michael Manganiello
045aa64558
cmd/copy: Prefix hostname to snapshot display output
...
This change better resembles the output generated by `Snapshot.String()`,
which includes both username and hostname.
Closes #4506
Before:
```
$ restic copy --from-repo /srv/restic-repo
repository 3666882b opened (version 2, compression level auto)
repository 0085c387 opened (version 2, compression level auto)
created new cache in /home/mike/.cache/restic
[0:00] 100.00% 1 / 1 index files loaded
[0:00] 0 index files loaded
snapshot 32b39a20 of [/home/mike/data] at 2023-10-21 16:01:13.979948154 -0300 -03)
copy started, this may take a while...
[0:00] 100.00% 1 / 1 packs copied
snapshot 10331fdd saved
```
After:
```
$ restic copy --from-repo /srv/restic-repo
repository 3666882b opened (version 2, compression level auto)
repository 0085c387 opened (version 2, compression level auto)
[0:00] 100.00% 1 / 1 index files loaded
[0:00] 0 index files loaded
snapshot 32b39a20 of [/home/mike/data] at 2023-10-21 16:01:13.979948154 -0300 -03 by mike@desktop)
copy started, this may take a while...
[0:00] 100.00% 1 / 1 packs copied
snapshot a67bd1ee saved
```
2024-01-06 20:20:46 +01:00
Michael Eischer
b2b7669ca0
Merge pull request #4526 from dnnr/detect-bitrot-in-diff
...
Add bitrot detection to "diff" command
2024-01-06 19:18:34 +00:00
Michael Eischer
4f6b1bb6f6
diff: document limitations regarding metadata
2024-01-06 20:12:47 +01:00
Michael Eischer
3549635243
diff: copy nodes before modifying them for bitrot detection
2024-01-06 20:12:47 +01:00
Daniel Danner
a7dc18e697
Add bitrot detection to "diff" command
...
This introduces a new modifier to the output of the diff command. It
appears whenever two files being compared only differ in their content
but not in their metadata. As far as we know, under normal
circumstances, this should only ever happen if some kind of bitrot has
happened in the source file. The prerequisite for this detection to work
is that the right-side snapshot of the comparison has been created with
"backup --force".
2024-01-06 20:12:47 +01:00
Michael Eischer
51419c51d3
archiver: Add filepath to error message if it is not included yet
2024-01-06 19:08:24 +01:00
Michael Eischer
6b79834cc8
archiver: improve error message for irregular files
...
Since Go 1.21, most reparse points are considered as irregular files.
Depending on the underlying driver these can exhibit nearly arbitrary
behavior. When encountering such a file, restic returned an
indecipherable error message: `error: invalid node type ""`.
Add the filepath to the error message and state that the file type is
not supported.
2024-01-06 19:03:11 +01:00
Michael Eischer
0018bb7854
restic: cleanup node type determination
...
os.ModeCharDevice is already included in os.ModeType
2024-01-06 18:43:16 +01:00
Michael Eischer
634e2a46d9
Merge pull request #4608 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.66
...
build(deps): bump github.com/minio/minio-go/v7 from 7.0.63 to 7.0.66
2024-01-06 11:40:02 +00:00
Michael Eischer
dfcab92db2
Merge pull request #4609 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.9.1
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.8.0 to 1.9.1
2024-01-06 11:06:07 +00:00
dependabot[bot]
3666eef76c
build(deps): bump github.com/minio/minio-go/v7 from 7.0.63 to 7.0.66
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.63 to 7.0.66.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.63...v7.0.66 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-01-06 11:03:52 +00:00
Michael Eischer
3a61622dfe
Merge pull request #4610 from restic/dependabot/github_actions/actions/setup-go-5
...
build(deps): bump actions/setup-go from 4 to 5
2024-01-06 11:01:07 +00:00
Michael Eischer
5c4fca76df
Merge pull request #4607 from restic/dependabot/go_modules/golang.org/x/crypto-0.17.0
...
build(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0
2024-01-06 10:56:40 +00:00
Michael Eischer
98da0bdd12
Merge pull request #4606 from restic/dependabot/go_modules/github.com/klauspost/compress-1.17.4
...
build(deps): bump github.com/klauspost/compress from 1.17.2 to 1.17.4
2024-01-06 10:55:00 +00:00
dependabot[bot]
2c60dd97ae
build(deps): bump actions/setup-go from 4 to 5
...
Bumps [actions/setup-go](https://github.com/actions/setup-go ) from 4 to 5.
- [Release notes](https://github.com/actions/setup-go/releases )
- [Commits](https://github.com/actions/setup-go/compare/v4...v5 )
---
updated-dependencies:
- dependency-name: actions/setup-go
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 01:33:20 +00:00
dependabot[bot]
40905403f4
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.8.0...sdk/azcore/v1.9.1 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 01:18:36 +00:00
dependabot[bot]
7e7cbe8e19
build(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0
...
Bumps [golang.org/x/crypto](https://github.com/golang/crypto ) from 0.16.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.16.0...v0.17.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/crypto
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 01:18:08 +00:00
dependabot[bot]
44646c20be
build(deps): bump github.com/klauspost/compress from 1.17.2 to 1.17.4
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.17.2 to 1.17.4.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.17.2...v1.17.4 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 01:17:57 +00:00
Michael Eischer
8f9a35779e
Merge pull request #4600 from MichaelEischer/doc-forget-note-1
...
Add a note that the oldest snapshot may be kept additionally
2023-12-29 17:56:54 +01:00
Michael Eischer
23e1b4bbb1
Merge pull request #4573 from gab50000/rewrite_time
...
Rewrite metadata
2023-12-24 14:42:00 +00:00
Michael Eischer
01b33734ab
rewrite: update command output in docs
2023-12-24 15:36:22 +01:00
Michael Eischer
649a6409ee
rewrite: cleanup tests
2023-12-24 15:36:22 +01:00
Michael Eischer
c31f5f986c
rewrite: Minor cleanups
2023-12-24 15:36:22 +01:00
Michael Eischer
2730d05fce
rewrite: Don't walk snapshot content if only metadata is modified
2023-12-24 15:36:22 +01:00
Michael Eischer
893d0d6325
rewrite: cleanup new metadata options and fix no parameters check
2023-12-24 15:36:22 +01:00
Gabriel Kabbe
7de97d7480
rewrite: Add documentation
2023-12-24 15:36:22 +01:00
Gabriel Kabbe
004520a238
rewrite: Add changelog
2023-12-24 15:36:22 +01:00
Gabriel Kabbe
a02d8d75c2
rewrite: Implement rewriting metadata
2023-12-24 15:36:22 +01:00
Gabriel Kabbe
7bf38b6c50
rewrite: Add test TestRewriteMetadata
2023-12-24 15:36:22 +01:00
Gabriel Kabbe
da1704b2d5
rewrite: Add tests
...
Pass nil instead of metadata to existing tests
2023-12-24 15:36:19 +01:00
Gabriel Kabbe
3026baea07
rewrite: Add structs for tracking metadata changes
...
Adds
* snapshotMetadataArgs, which holds the new metadata as strings parsed from
the command line
* snapshotMetadata, which holds the new metadata converted to the
correct types
2023-12-24 14:43:07 +01:00
Michael Eischer
1196c72819
Merge pull request #4570 from MarkusZoppelt/docs/pkgx
...
docs: add pkgx install option
2023-12-24 11:04:59 +00:00
Michael Eischer
433dd92959
Merge pull request #4590 from renard/optimize-mount-failure
...
mount: detect mountpoint does not exist before opening the repository
2023-12-24 10:59:56 +00:00
Markus Zoppelt
c14740c50f
docs: add pkgx install option
...
PR in pkgx pantry: https://github.com/pkgxdev/pantry/pull/4098
restic pkg:
https://pkgx.dev/pkgs/restic.net/restic/
2023-12-24 11:59:12 +01:00
Michael Eischer
5537460664
Merge pull request #4598 from MichaelEischer/add_table_of_contents
...
Add table of contents
2023-12-24 11:56:05 +01:00
Sébastien Gross
f7587be28f
mount: detect mountpoint does not exist before opening the repository
...
Bug #1681 suggests that restic should not be nice to user and should
refrain from creating a mountpoint if it does not exist. Nevertheless,
it currently opens the repository before checking for the mountpoint's
existence. In the case of large or remote repositories, this process
can be time-consuming, delaying the inevitable outcome.
/restic mount --repo=REMOTE --verbose /tmp/backup
repository 33f14e42 opened (version 2, compression level max)
[0:38] 100.00% 162 / 162 index files loaded
Mountpoint /tmp/backup doesn't exist
stat /tmp/backup: no such file or directory
real 0m39.534s
user 1m53.961s
sys 0m3.044s
In this scenario, 40 seconds could have been saved if the nonexistence
of the path had been verified beforehand.
This patch relocates the mountpoint check to the beginning of the
runMount function, preceding the opening of the repository.
/restic mount --repo=REMOTE --verbose /tmp/backup
Mountpoint /tmp/backup doesn't exist
stat /tmp/backup: no such file or directory
real 0m0.136s
user 0m0.018s
sys 0m0.027s
Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
2023-12-24 11:54:18 +01:00
Michael Eischer
91fb703756
regenerate changelog
2023-12-24 11:47:31 +01:00
Michael Eischer
d7ff862b8d
cleanup changelog whitespace
2023-12-24 11:47:31 +01:00
mmattel
db1d920c80
Add a table of contents (TOC) to the changelog template
2023-12-24 11:47:31 +01:00
Michael Eischer
c6299f8dbd
Merge pull request #4582 from giuseppedandrea/docs/fix-typo
...
docs: fix typo in working with repos
2023-12-24 00:09:59 +00:00
Giuseppe D'Andrea
a128976014
docs: fix typo in working with repos
...
When using the `copy` command, `--from-password-file` and `--from-password-command` flags are used to specify the password of the source repository, not of the destination repository.
2023-12-24 01:04:36 +01:00
Michael Eischer
e2f6109a52
Merge pull request #4580 from restic/dependabot/go_modules/golang.org/x/oauth2-0.15.0
...
build(deps): bump golang.org/x/oauth2 from 0.13.0 to 0.15.0
2023-12-24 00:02:30 +00:00
Michael Eischer
30e6ed038c
Merge pull request #4586 from Gelma/typos
...
Fix typos
2023-12-23 13:21:50 +00:00
Michael Eischer
e96d1ee33e
Merge pull request #4593 from SimJoSt/patch-1
...
docs(scripting): correct stats output comment to be about the correct command
2023-12-23 13:09:20 +00:00
dependabot[bot]
0054db394f
build(deps): bump golang.org/x/oauth2 from 0.13.0 to 0.15.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.13.0 to 0.15.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.13.0...v0.15.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-12-23 13:08:29 +00:00
Michael Eischer
53ebe91a50
Move changelog typo fixes to changelog files
2023-12-23 14:05:47 +01:00
Michael Eischer
9ceaea34dd
Merge pull request #4572 from linktohack/patch-1
...
docs: Mention progress for restore command. This is available after https://github.com/restic/restic/pull/3991
2023-12-23 13:00:32 +00:00
Michael Eischer
356b7aac16
Merge pull request #4571 from michaelkuhn/init-repo-file
...
Fix repository not being printed when using repository file
2023-12-23 12:59:06 +00:00
Joda Stößer
eef7c65655
docs(scripting): correct stats output comment to be about the correct command
...
not about the snapshots command
2023-12-23 13:56:40 +01:00
Michael Eischer
97b8629336
Merge pull request #4579 from restic/dependabot/go_modules/golang.org/x/sync-0.5.0
...
build(deps): bump golang.org/x/sync from 0.4.0 to 0.5.0
2023-12-23 12:51:52 +00:00
Michael Eischer
d2ecd6bef2
Merge pull request #4577 from restic/dependabot/go_modules/golang.org/x/time-0.5.0
...
build(deps): bump golang.org/x/time from 0.3.0 to 0.5.0
2023-12-23 12:50:59 +00:00
Michael Eischer
634750a732
Merge pull request #4576 from restic/dependabot/github_actions/docker/login-action-3d58c274f17dffee475a5520cbe67f0a882c4dbb
...
build(deps): bump docker/login-action from 1f401f745bf57e30b3a2800ad308a87d2ebdf14b to 3d58c274f17dffee475a5520cbe67f0a882c4dbb
2023-12-23 12:50:52 +00:00
Quang-Linh LE
c554825e2d
docs: Mention progress for restore command. This is available after https://github.com/restic/restic/pull/3991
2023-12-23 13:50:20 +01:00
Michael Kuhn
fd2fb233aa
Fix repository not being printed when using repository file
...
When using `RESTIC_REPOSITORY_FILE` in combination with `restic init`,
the repository is missing in the output:
```
$ restic init
created restic repository 3c872be20f at
[...]
```
This is due to the code using `gopts.Repo`, which is empty in this case.
2023-12-23 13:49:22 +01:00
Michael Eischer
da4e3edbbc
Merge pull request #4596 from MichaelEischer/update-golangcilint
...
CI: update golangci-lint
2023-12-23 13:48:33 +01:00
Michael Eischer
dbbd31bc3a
CI: update golangci-lint
...
Necessary to properly support Go 1.21.
2023-12-23 13:41:30 +01:00
Joram Berger
12af20e606
Add a note that the oldest snapshot may be kept additionally
...
Documentation enhancement.
2023-12-18 18:24:57 +01:00
Andrea Gelmini
241916d55b
Fix typos
2023-12-06 13:11:55 +01:00
dependabot[bot]
427b90cf82
build(deps): bump golang.org/x/sync from 0.4.0 to 0.5.0
...
Bumps [golang.org/x/sync](https://github.com/golang/sync ) from 0.4.0 to 0.5.0.
- [Commits](https://github.com/golang/sync/compare/v0.4.0...v0.5.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sync
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-12-01 01:15:09 +00:00
dependabot[bot]
eec6e014f4
build(deps): bump golang.org/x/time from 0.3.0 to 0.5.0
...
Bumps [golang.org/x/time](https://github.com/golang/time ) from 0.3.0 to 0.5.0.
- [Commits](https://github.com/golang/time/compare/v0.3.0...v0.5.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/time
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-12-01 01:14:53 +00:00
dependabot[bot]
fa46a47e22
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from 1f401f745bf57e30b3a2800ad308a87d2ebdf14b to 3d58c274f17dffee475a5520cbe67f0a882c4dbb.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](1f401f745b...3d58c274f1
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-12-01 01:02:16 +00:00
Michael Eischer
b72de5a883
Merge branch 'patch-release'
2023-11-12 11:52:21 +01:00
Michael Eischer
6086ae4ca7
Merge pull request #4563 from smlx/mac-signature
...
fix: drop reference to signature and define MAC
2023-11-08 21:59:13 +00:00
Scott Leggett
aeaf527be1
fix: drop reference to signature and define MAC
...
Poly1305-AES is not a signature, so don't mention that.
In addition, the term MAC was used without being defined, so add a
definition.
Signed-off-by: Scott Leggett <scott@sl.id.au>
2023-11-06 20:12:42 +08:00
Michael Eischer
19068aa82f
Merge pull request #4559 from DRON-666/html-zip
...
Restore generation of `HTMLZip` files on `readthedocs.com`.
2023-11-02 20:53:12 +00:00
Michael Eischer
81ca9d28f2
Merge pull request #4553 from CommanderRoot/add-version-json
...
Add --json option to version command
2023-11-01 21:58:38 +00:00
Tobias Speicher
ce53ea32c6
Split `go_target` into `go_os` and `go_arch`
2023-11-01 22:43:38 +01:00
Tobias Speicher
10cbc169c1
Use different function to be more consistent with other code
2023-11-01 22:18:37 +01:00
DRON-666
03f8f494e9
doc: add HTMLZip format to .readthedocs.yaml
2023-11-02 00:16:47 +03:00
Tobias Speicher
ab23d033b6
Add version command output to JSON format documentation
2023-11-01 22:13:57 +01:00
Tobias Speicher
6f1efcb28b
Update wording on changelog entry
2023-11-01 22:12:19 +01:00
Michael Eischer
9c399e55e3
Merge pull request #4554 from restic/dependabot/github_actions/docker/login-action-1f401f745bf57e30b3a2800ad308a87d2ebdf14b
...
build(deps): bump docker/login-action from b4bedf8053341df3b5a9f9e0f2cf4e79e27360c6 to 1f401f745bf57e30b3a2800ad308a87d2ebdf14b
2023-11-01 20:42:11 +00:00
Michael Eischer
e550bc0713
Merge pull request #4555 from restic/dependabot/go_modules/google.golang.org/api-0.149.0
...
build(deps): bump google.golang.org/api from 0.148.0 to 0.149.0
2023-11-01 20:40:45 +00:00
Michael Eischer
28aa9826af
Merge pull request #4557 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.34.0
...
build(deps): bump cloud.google.com/go/storage from 1.33.0 to 1.34.0
2023-11-01 20:40:40 +00:00
dependabot[bot]
6dde019ac8
build(deps): bump cloud.google.com/go/storage from 1.33.0 to 1.34.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.33.0 to 1.34.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.33.0...spanner/v1.34.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-11-01 01:45:02 +00:00
dependabot[bot]
c2f9e21d3c
build(deps): bump google.golang.org/api from 0.148.0 to 0.149.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.148.0 to 0.149.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.148.0...v0.149.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-11-01 01:44:28 +00:00
dependabot[bot]
67e6b9104a
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from b4bedf8053341df3b5a9f9e0f2cf4e79e27360c6 to 1f401f745bf57e30b3a2800ad308a87d2ebdf14b.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](b4bedf8053...1f401f745b
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-11-01 01:25:22 +00:00
Tobias Speicher
6ca07ee004
add changelog/unreleases for issue-4547
2023-10-31 19:39:52 +01:00
Tobias Speicher
d45cc52468
command version: add json output option
2023-10-31 19:31:07 +01:00
Alexander Neumann
59fe24cb2b
Set development version for 0.16.2
2023-10-29 19:50:12 +01:00
Alexander Neumann
1a5efcf680
Add version for 0.16.2
2023-10-29 19:50:07 +01:00
Alexander Neumann
d33fe6dd3c
Generate CHANGELOG.md for 0.16.2
2023-10-29 19:50:04 +01:00
Alexander Neumann
c8dd95f104
Prepare changelog for 0.16.2
2023-10-29 19:50:04 +01:00
Leo R. Lundgren
7d980b469d
doc: Polish changelogs
2023-10-29 00:44:53 +02:00
Michael Eischer
d863234e3e
add changelog for missing documentation
2023-10-28 20:25:24 +02:00
Michael Eischer
42ab3ea2b9
Merge pull request #4410 from Enrico204/restic-stdin-command
...
add --stdin-from-command flag to backup command
2023-10-27 22:09:06 +00:00
Michael Eischer
be28a02626
doc: tweak description for --stdin-from-command
2023-10-27 23:58:52 +02:00
Michael Eischer
5d152c7720
extend changelog for --stdin-from-command
2023-10-27 23:58:52 +02:00
Michael Eischer
ee305e6041
backup: rework error reporting for subcommand
2023-10-27 23:58:52 +02:00
Michael Eischer
8bceb8e359
fs: add tests for CommandReader
2023-10-27 23:58:52 +02:00
Michael Eischer
317144c1d6
fs: merge command startup into CommandReader
2023-10-27 23:58:51 +02:00
Michael Eischer
7d879705ad
fs: cleanup CommandReader implementation
2023-10-27 23:58:51 +02:00
Enrico204
37a312e505
restic-from-command: use standard behavior when no output and exit code 0 from command
...
The behavior of the new option should reflect the behavior of normal backups: when the command exit code is zero and there is no output in the stdout, emit a warning but create the snapshot. This commit fixes the integration tests and the ReadCloserCommand struct.
2023-10-27 23:58:51 +02:00
Enrico204
c0ca54dc8a
restic-from-command: add tests
2023-10-27 23:58:51 +02:00
Enrico204
81f8d473df
restic-from-command: abort snapshot on non-zero exit codes
2023-10-27 23:58:51 +02:00
Enrico204
6990b0122e
Add issue-4251 (stdin-from-command) in the changelog
2023-10-27 23:58:51 +02:00
Enrico204
072b227544
stdin-from-command: add documentation in backup sub-command
2023-10-27 23:58:51 +02:00
Enrico204
4e5caab114
stdin-from-command: implemented suggestions in #4254
...
The code has been refactored so that the archiver is back to the original code, and the stderr is handled using a go routine to avoid deadlock.
2023-10-27 23:58:51 +02:00
Sebastian Hoß
c133065a9f
Check command result before snapshotting
...
Return with an error containing the stderr of the given command in case it fails. No new snapshot will be created and future prune operations on the repository will remove the unreferenced data.
Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
25350a9c55
Extend SnapshotOptions w/ command data
...
In order to determine whether to save a snapshot, we need to capture the exit code returned by a command. In order to provide a nice error message, we supply stderr as well.
Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
a2b76ff34f
Start command from --stdin-from-command
...
It acts similar to --stdin but reads its data from the stdout of the given command instead of os.Stdin.
Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
333fe1c3cf
Align Stdin and StdinCommand in conditionals
...
In order to run with --stdin-from-command we need to short-circuit some functions similar to how it is handled for the --stdin flag. The only difference here is that --stdin-from-command actually expects that len(args) should be greater 0 whereas --stdin does not expect any args at all.
Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
a8657bde68
Add --stdin-from-command option
...
This new flag is added to the backup subcommand in order to allow restic to control the execution of a command and determine whether to save a snapshot if the given command succeeds.
Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Michael Eischer
104107886a
Merge pull request #4503 from MichaelEischer/fix-stats-with-hardlinks
...
stats: Fix hardlink tracking across multiple filesystems
2023-10-27 23:53:03 +02:00
Michael Eischer
731b3a4357
stats: fix hardlink tracking in a snapshot
...
inodes are only unique within a device. Use the HardlinkIndex from the
restorer instead of the custom (broken) hashmap to correctly account for
both inode and deviceID.
2023-10-27 23:40:42 +02:00
Michael Eischer
a8fdcf79b7
restorer: Make hardlink index generic
...
This will allow reusing it for the stats command without regressing the
memory usage due to storing an unnecessary file path.
2023-10-27 23:40:42 +02:00
Michael Eischer
45962c2847
Merge pull request #4499 from MichaelEischer/modular-backend-code
...
Split backend code from restic package
2023-10-27 20:19:20 +02:00
Michael Eischer
4be45de1c2
Restore support for ARMv5 platforms
2023-10-27 19:52:14 +02:00
Leo R. Lundgren
8c1125fe13
doc: Correct two typos
2023-10-27 19:48:00 +02:00
Martin Michlmayr
0b6ccea461
Fix typos
2023-10-27 19:48:00 +02:00
Martin Michlmayr
de6135351e
Format option correctly
2023-10-27 19:48:00 +02:00
Michael Eischer
d47581b25e
verify-release-binaries.sh: don't show warning if binaries are correct
2023-10-27 19:48:00 +02:00
Michael Eischer
69dec02a14
Remove readthedocs special case from docs configuration
...
Apparently it's now required to bring your own theme.
2023-10-27 19:47:19 +02:00
Michael Eischer
826d880614
Fix doc path typo in readthedocs configuration
2023-10-27 19:47:15 +02:00
Michael Eischer
dbf7ef72b9
Add read the docs config file version 2
...
The config file is by now necessary to build documentation:
https://blog.readthedocs.com/migrate-configuration-v2/
2023-10-27 19:47:11 +02:00
Michael Eischer
50ef01131a
Merge pull request #4542 from MichaelEischer/support-armv6
...
Only support ARMv6 on ARM platforms
2023-10-27 19:42:49 +02:00
rawtaz
6be3a8fe51
Merge pull request #4539 from tbm/docs
...
Fix typos in docs
2023-10-27 17:30:08 +00:00
Michael Eischer
5166bde386
Only support ARMv6 on ARM platforms
...
Go 1.21 has switched the default from GOARM=5 to GOARM=7. So far there
have been complaints from Raspberry Pi 1 users, as the first raspberry
pi version only supports ARMv6. Exclude older ARM versions as these are
likely not relevant (rest-server also only supports ARMv6/7) and enforce
the use of software floating point emulation.
2023-10-27 19:12:12 +02:00
Leo R. Lundgren
aafb806a8c
doc: Correct two typos
2023-10-27 18:56:32 +02:00
Martin Michlmayr
41e6a02bcc
Fix typos
2023-10-27 18:56:32 +02:00
Martin Michlmayr
b51fe2fb69
Format option correctly
2023-10-27 18:56:32 +02:00
Michael Eischer
56537fb48e
Merge pull request #4545 from restic/fix-rtd3
...
Try to fix documentation build
2023-10-26 22:00:20 +02:00
Michael Eischer
feea567868
Remove readthedocs special case from docs configuration
...
Apparently it's now required to bring your own theme.
2023-10-26 21:56:36 +02:00
Michael Eischer
2968b52f84
Merge pull request #4543 from MichaelEischer/fix-rtd2
...
Fix doc path typo in readthedocs configuration
2023-10-26 20:40:45 +02:00
Michael Eischer
619e80d7cc
Fix doc path typo in readthedocs configuration
2023-10-26 20:39:43 +02:00
Michael Eischer
3804e50d64
Merge pull request #4541 from MichaelEischer/fix-binary-check-script
...
verify-release-binaries.sh: don't show warning if binaries are correct
2023-10-26 20:37:56 +02:00
Michael Eischer
c19e39968f
verify-release-binaries.sh: don't show warning if binaries are correct
2023-10-26 19:59:27 +02:00
Michael Eischer
550be5c1e9
Merge pull request #4538 from MichaelEischer/fix-rtd
...
Add read the docs config file version 2
2023-10-26 19:42:36 +02:00
Michael Eischer
249605843b
prune: get backend connection count via repository
2023-10-25 23:01:54 +02:00
Michael Eischer
c7b770eb1f
convert MemorizeList to be repository based
...
Ideally, code that uses a repository shouldn't directly interact with
the underlying backend. Thus, move MemorizeList one layer up.
2023-10-25 23:01:35 +02:00
Michael Eischer
1b8a67fe76
move Backend interface to backend package
2023-10-25 23:00:18 +02:00
Michael Eischer
ceb0774af1
backend: make LoadAll independent of restic package
2023-10-25 22:58:39 +02:00
Michael Eischer
b6d79bdf6f
restic: decouple restic.Handle
2023-10-25 22:54:07 +02:00
Michael Eischer
7881309d63
backend: move backend implementation helpers to util package
...
This removes code that is only used within a backend implementation from
the backend package. The latter now only contains code that also has
external users.
2023-10-25 22:54:07 +02:00
Michael Eischer
8e6fdf5edf
Merge pull request #4520 from awannabeengineer/load-retry-nonexistent
...
retry: Do not retry Load() if file does not exist
2023-10-25 20:42:05 +00:00
Michael Eischer
c2e3e8d6ea
Add read the docs config file version 2
...
The config file is by now necessary to build documentation:
https://blog.readthedocs.com/migrate-configuration-v2/
2023-10-25 22:00:42 +02:00
Alexander Neumann
27ec320eae
Set development version for 0.16.1
2023-10-24 20:02:58 +02:00
Alexander Neumann
baca3f6303
Add version for 0.16.1
2023-10-24 20:02:55 +02:00
Alexander Neumann
524c2721b4
Update manpages and auto-completion
2023-10-24 20:02:55 +02:00
Alexander Neumann
be1b978ac8
Generate CHANGELOG.md for 0.16.1
2023-10-24 20:02:29 +02:00
Alexander Neumann
d4d9c1b8f1
Prepare changelog for 0.16.1
2023-10-24 20:02:29 +02:00
Michael Eischer
ead8dd0173
Merge pull request #4524 from MichaelEischer/fix-autocompletion
...
Don't load password when autocompleting CLI commands
2023-10-23 20:06:53 +00:00
Michael Eischer
cd09ef4614
don't load password when autocompleting CLI commands
2023-10-23 21:44:22 +02:00
Michael Eischer
d399e32590
Merge pull request #4535 from MichaelEischer/fix-cat-command
...
Only show index loading progress bar if stdout is a terminal
2023-10-23 21:28:13 +02:00
Michael Eischer
54a4034ec0
Only show index progress bar if stdout is a terminal
...
This ensures that there is no interference when using restic for
scripting use cases.
2023-10-23 21:16:52 +02:00
rawtaz
138b7b3328
Merge pull request #4534 from MichaelEischer/polish-changelogs
...
Polish changelogs
2023-10-23 19:00:01 +00:00
Leo R. Lundgren
6d19e0260d
doc: Polish changelogs
2023-10-23 20:45:15 +02:00
Michael Eischer
85abceb99c
doc: update minimum required Go version
2023-10-23 19:02:20 +02:00
Michael Eischer
85c15e6fa3
polish changelogs
2023-10-23 19:01:45 +02:00
Michael Eischer
d6917c7e00
Merge pull request #4533 from MichaelEischer/update-dependencies
...
update dependencies
2023-10-23 19:01:03 +02:00
Michael Eischer
8c20301172
update dependencies
2023-10-23 18:48:13 +02:00
Michael Eischer
4b4f916bdc
Merge pull request #4532 from MichaelEischer/update-zstd
...
Update klauspost/compress to fix data corruption at maximum compression
2023-10-23 18:45:35 +02:00
Michael Eischer
9707956375
Merge pull request #4530 from MichaelEischer/repair-packs
...
Add `repair packs` command to simplify recovery from damaged pack
2023-10-23 18:44:42 +02:00
Michael Eischer
d1d4510974
repair packs: Add stub feature flag implementation
2023-10-23 18:36:28 +02:00
Michael Eischer
a28940ea29
check: Suggest usage of `restic repair packs` for corrupted blobs
...
For now, the guide is only shown if the blob content does not match its
hash. The main intended usage is to handle data corruption errors when
using maximum compression in restic 0.16.0
2023-10-23 18:36:28 +02:00
Michael Eischer
db26dc75e1
repair packs: add experimental command
...
This allows recovering a repository from several cases of damaged blobs.
2023-10-23 18:36:28 +02:00
Michael Eischer
efef38d0e8
drop support for Go 1.18
2023-10-23 18:30:38 +02:00
Michael Eischer
d00e72fed4
Update klauspost/compress to fix data corruption at maximum compression
2023-10-23 18:30:38 +02:00
Arash Farr
d15ffd9c92
retry: Do not retry Load() if file does not exist
2023-10-22 13:25:32 -05:00
Michael Eischer
62af0d769a
Merge pull request #4528 from adamantike/misc/run-tests-with-go-1.21
...
Run tests with Go 1.21
2023-10-22 16:00:27 +00:00
Michael Eischer
ae83a9002a
Merge pull request #4531 from MichaelEischer/silence-automaxprocs
...
Disable log output from automaxprocs library
2023-10-22 17:43:36 +02:00
Michael Eischer
ceff4af1ac
Disable log output from automaxprocs library
2023-10-22 17:09:46 +02:00
Michael Manganiello
b15ba553a4
Run tests with Go 1.21
2023-10-22 10:47:42 -03:00
Michael Eischer
46cb1df1bc
Merge pull request #4505 from MichaelEischer/verify-release-binaries
...
Add script to verify release binaries
2023-10-21 20:00:25 +02:00
Michael Eischer
bd3816fa14
CI: Ensure that github containers match the official binaries
...
The binaries accidentally included VCS information whereas binaries
built from the release tarball do not. For consistency remove the .git
directory before building the container on Github.
2023-10-21 19:44:59 +02:00
Michael Eischer
b2b0856908
Add helper script to verify release binaries
...
The script checks that the released binaries and the container binaries
can be reproduced.
2023-10-21 19:44:59 +02:00
Michael Eischer
7f05af02b9
Merge pull request #4480 from AgathaSorceress/add-rest-auth-env
...
Support reading basic auth credentials for REST server from environment variables
2023-10-21 17:41:08 +00:00
Michael Eischer
eabc177a42
Merge pull request #4442 from Refutable4890/copy-doc
...
Add a description of the resumption of `copy`
2023-10-21 17:38:15 +00:00
Michael Eischer
ab6defbace
Merge pull request #4519 from adamantike/backend/sftp/add-args-option
...
backend/sftp: Add sftp.args option
2023-10-21 17:37:46 +00:00
Michael Eischer
fe1f61570b
Merge pull request #4518 from adamantike/backend/b2/switch-to-official-library
...
backend/b2: Switch to official library `Backblaze/blazer`
2023-10-21 17:29:19 +00:00
Michael Eischer
baf9b54891
sftp: simplify documentation for sftp.args
2023-10-21 19:28:27 +02:00
Michael Eischer
6a4d6d5da4
sftp: test that Args and Command options cannot be set at the same time
2023-10-21 19:26:39 +02:00
Michael Manganiello
41f70f1f4f
backend/sftp: Add sftp.args option
...
Allow setting custom arguments for the `sftp` backend, by using the
`sftp.args` option. This is similar to the approach already implemented
in the `rclone` backend, to support new arguments without requiring
future code changes for each different SSH argument.
Closes #4241
2023-10-21 19:25:44 +02:00
Michael Eischer
6c7560e537
backend/b2: Switch to official library `Backblaze/blazer`
...
According to https://github.com/kurin/blazer Readme, the repository is
no longer maintained, and the official one at
https://github.com/Backblaze/blazer took over.
Closes #4148
2023-10-21 19:18:13 +02:00
Agatha V. Lovelace
0f97356b21
docs: add new REST server env variables
2023-10-20 13:04:58 +02:00
Agatha V. Lovelace
2089c54310
changelog: add unreleased entry
2023-10-20 13:04:58 +02:00
Agatha V. Lovelace
f1877e721e
feat: support reading REST credentials from env
2023-10-20 13:04:58 +02:00
Michael Eischer
17f2301cc2
Merge pull request #4514 from grembo/key-list-no-lock
...
Make `key list` command honor `--no-lock`
2023-10-19 20:50:56 +00:00
Michael Eischer
79deb99605
Merge pull request #4500 from MichaelEischer/simplify-issue-template
...
Simplify issue template
2023-10-19 20:48:36 +00:00
Michael Eischer
643180b415
simplify issue template
2023-10-19 22:16:38 +02:00
Michael Eischer
d27cfd10a9
Merge pull request #4487 from lbausch/tweak-restore-summary
...
Tweak restore summary
2023-10-19 20:14:14 +00:00
Michael Gmelin
34f3b13b7c
Make `key list` command honor `--no-lock`
...
Fixes #4513
2023-10-19 22:09:35 +02:00
Michael Eischer
495982232c
Merge pull request #4511 from dnnr/inode-in-json
...
Include inode in JSON output for find and ls
2023-10-17 21:06:34 +00:00
Michael Eischer
d173d1c532
Merge pull request #4486 from MichaelEischer/separate-large-blobs
...
repository: store oversized blobs in separate pack files
2023-10-17 23:03:52 +02:00
Michael Eischer
f955222750
Merge pull request #4502 from MichaelEischer/replace-ratelimit
...
backend/limiter: replace juju/ratelimit with x/time/rate
2023-10-17 23:03:17 +02:00
Michael Eischer
cb9cbe55d9
repository: store oversized blobs in separate pack files
...
Store oversized blobs in separate pack files as the blobs is large
enough to warrant its own pack file. This simplifies the garbage
collection of such blobs and keeps the cache smaller, as oversize (tree)
blobs only have to be downloaded if they are actually used.
2023-10-17 22:52:16 +02:00
Michael Eischer
f750aa8dfb
backend/limiter: replace juju/ratelimit with x/time/rate
2023-10-17 22:49:25 +02:00
Michael Eischer
c635e30e3f
Merge pull request #4485 from MichaelEischer/uber-automaxprocs
...
Automatically adjust GOMAXPROCS when run in a cpu-constrained container
2023-10-17 22:47:10 +02:00
Michael Eischer
f5d5e8fd0a
automatically adjust GOMAXPROCS when run in a cpu-constrained container
2023-10-17 20:31:57 +02:00
Daniel Danner
a858ab254b
Include inode in JSON output for find and ls
...
This introduces the inode attribute to the JSON output emitted for nodes
in `ls` and matches in `find`. There doesn't seem to be any discernible
reason to omit the inode and it can be useful in scripting scenarios.
2023-10-11 21:10:01 +02:00
Michael Eischer
4087c3aff7
Merge pull request #4498 from PeterDaveHello/ImproveDockerfile
...
Remove additional `--update` for apk in Dockerfile
2023-10-09 20:53:49 +00:00
Refutable4890
aa86c76aab
Add a description of the resumption of `copy`
2023-10-08 12:07:40 +08:00
Michael Eischer
76ef94d15c
Merge pull request #4419 from arjunajesh/issue#299
...
Progress Bar when loading index files
2023-10-01 18:06:35 +00:00
Michael Eischer
91aef00df3
check: add index loading progress bar
2023-10-01 19:55:29 +02:00
Michael Eischer
b0da0f152f
deduplicate index progress bar setup
2023-10-01 19:53:26 +02:00
Michael Eischer
75f6bd89ed
hide index loading progress for JSON output
2023-10-01 19:53:26 +02:00
Michael Eischer
3fd0ad7448
repository: list index files only once
2023-10-01 19:53:26 +02:00
arjunajesh
b6593ad7df
changelog
2023-10-01 19:53:26 +02:00
arjunajesh
ed65a7dbca
implement progress bar for index loading
2023-10-01 19:52:59 +02:00
Michael Eischer
eac1c4a8d0
Merge pull request #4471 from jalseth/master
...
docs(examples): Use restic binary from system path
2023-10-01 12:31:28 +00:00
Michael Eischer
f519454f33
Merge pull request #4493 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.33.0
...
build(deps): bump cloud.google.com/go/storage from 1.31.0 to 1.33.0
2023-10-01 12:09:43 +00:00
Michael Eischer
54ae8a0c40
Merge pull request #4492 from restic/dependabot/github_actions/docker/login-action-b4bedf8053341df3b5a9f9e0f2cf4e79e27360c6
...
build(deps): bump docker/login-action from cf8514a65188af1d4f94f8c28a7a4153af1088ce to b4bedf8053341df3b5a9f9e0f2cf4e79e27360c6
2023-10-01 12:08:10 +00:00
Michael Eischer
a36b5b6391
Merge pull request #4491 from restic/dependabot/github_actions/docker/setup-buildx-action-3
...
build(deps): bump docker/setup-buildx-action from 2 to 3
2023-10-01 12:03:48 +00:00
Michael Eischer
5e36e4da96
Merge pull request #4489 from restic/dependabot/github_actions/docker/setup-qemu-action-3
...
build(deps): bump docker/setup-qemu-action from 2 to 3
2023-10-01 12:03:19 +00:00
Michael Eischer
8ee08e5d09
Merge pull request #4490 from restic/dependabot/github_actions/actions/checkout-4
...
build(deps): bump actions/checkout from 3 to 4
2023-10-01 12:03:09 +00:00
Michael Eischer
9f9f736ec2
Merge pull request #4495 from restic/dependabot/go_modules/golang.org/x/oauth2-0.12.0
...
build(deps): bump golang.org/x/oauth2 from 0.11.0 to 0.12.0
2023-10-01 12:01:12 +00:00
Michael Eischer
af98c3ccbe
Merge pull request #4496 from restic/dependabot/go_modules/github.com/hashicorp/golang-lru/v2-2.0.7
...
build(deps): bump github.com/hashicorp/golang-lru/v2 from 2.0.6 to 2.0.7
2023-10-01 11:59:38 +00:00
Peter Dave Hello
6edfc73879
Remove additional `--update` for apk in Dockerfile
2023-10-01 18:00:15 +08:00
dependabot[bot]
d985ed27d1
build(deps): bump github.com/hashicorp/golang-lru/v2 from 2.0.6 to 2.0.7
...
Bumps [github.com/hashicorp/golang-lru/v2](https://github.com/hashicorp/golang-lru ) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/hashicorp/golang-lru/releases )
- [Commits](https://github.com/hashicorp/golang-lru/compare/v2.0.6...v2.0.7 )
---
updated-dependencies:
- dependency-name: github.com/hashicorp/golang-lru/v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:45:19 +00:00
dependabot[bot]
4278ec6553
build(deps): bump golang.org/x/oauth2 from 0.11.0 to 0.12.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.11.0 to 0.12.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.11.0...v0.12.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:45:07 +00:00
dependabot[bot]
15cb498c47
build(deps): bump cloud.google.com/go/storage from 1.31.0 to 1.33.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.31.0 to 1.33.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.31.0...pubsub/v1.33.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:44:54 +00:00
dependabot[bot]
59e217b003
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from cf8514a65188af1d4f94f8c28a7a4153af1088ce to b4bedf8053341df3b5a9f9e0f2cf4e79e27360c6.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](cf8514a651...b4bedf8053
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:24:15 +00:00
dependabot[bot]
3457f50c8c
build(deps): bump docker/setup-buildx-action from 2 to 3
...
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action ) from 2 to 3.
- [Release notes](https://github.com/docker/setup-buildx-action/releases )
- [Commits](https://github.com/docker/setup-buildx-action/compare/v2...v3 )
---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:24:10 +00:00
dependabot[bot]
bbe2f1ecf2
build(deps): bump actions/checkout from 3 to 4
...
Bumps [actions/checkout](https://github.com/actions/checkout ) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases )
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md )
- [Commits](https://github.com/actions/checkout/compare/v3...v4 )
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:24:05 +00:00
dependabot[bot]
362917afb9
build(deps): bump docker/setup-qemu-action from 2 to 3
...
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action ) from 2 to 3.
- [Release notes](https://github.com/docker/setup-qemu-action/releases )
- [Commits](https://github.com/docker/setup-qemu-action/compare/v2...v3 )
---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:24:01 +00:00
Michael Eischer
b92ab458b0
Merge pull request #4459 from restic/dependabot/go_modules/github.com/go-ole/go-ole-1.3.0
...
build(deps): bump github.com/go-ole/go-ole from 1.2.6 to 1.3.0
2023-09-28 20:42:14 +00:00
Lorenz Bausch
2657217574
Tweak restore summary
2023-09-20 10:59:55 +02:00
James Alseth
02ab511c2f
docs(examples): Use restic binary from system path
...
Signed-off-by: James Alseth <james@jalseth.me>
2023-09-17 19:39:45 -07:00
Michael Eischer
6e586b64e4
Merge pull request #4450 from MichaelEischer/cleanup-master-index-test
...
master_index: fix inconsistent length blob length in test
2023-09-06 22:55:48 +02:00
Michael Eischer
fb4d458cce
Merge pull request #4452 from lbausch/docs/add-current-key-hint
...
docs: explain meaning of asterisk when managing repository keys
2023-09-06 20:28:33 +00:00
Michael Eischer
c7b5ddc012
Merge pull request #4451 from greatroar/bh-string
...
internal/restic: Return summary from BlobSet.String
2023-09-06 20:27:26 +00:00
Michael Eischer
3eb825e47c
Merge pull request #4464 from restic/dependabot/go_modules/golang.org/x/term-0.12.0
...
build(deps): bump golang.org/x/term from 0.11.0 to 0.12.0
2023-09-06 20:18:34 +00:00
dependabot[bot]
4d60011030
build(deps): bump golang.org/x/term from 0.11.0 to 0.12.0
...
Bumps [golang.org/x/term](https://github.com/golang/term ) from 0.11.0 to 0.12.0.
- [Commits](https://github.com/golang/term/compare/v0.11.0...v0.12.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/term
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-09-05 21:43:28 +00:00
Michael Eischer
507ed32469
Merge pull request #4457 from restic/dependabot/go_modules/google.golang.org/api-0.138.0
...
build(deps): bump google.golang.org/api from 0.134.0 to 0.138.0
2023-09-05 21:32:34 +00:00
dependabot[bot]
22fdfe1ffe
build(deps): bump github.com/go-ole/go-ole from 1.2.6 to 1.3.0
...
Bumps [github.com/go-ole/go-ole](https://github.com/go-ole/go-ole ) from 1.2.6 to 1.3.0.
- [Release notes](https://github.com/go-ole/go-ole/releases )
- [Changelog](https://github.com/go-ole/go-ole/blob/master/ChangeLog.md )
- [Commits](https://github.com/go-ole/go-ole/compare/v1.2.6...v1.3.0 )
---
updated-dependencies:
- dependency-name: github.com/go-ole/go-ole
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-09-05 21:17:03 +00:00
Michael Eischer
e05cd9abca
Merge pull request #4462 from restic/dependabot/github_actions/docker/login-action-cf8514a65188af1d4f94f8c28a7a4153af1088ce
...
build(deps): bump docker/login-action from a9794064588be971151ec5e7144cb535bcb56e36 to cf8514a65188af1d4f94f8c28a7a4153af1088ce
2023-09-05 21:07:57 +00:00
Michael Eischer
ea55ca5303
Merge pull request #4458 from restic/dependabot/go_modules/github.com/hashicorp/golang-lru/v2-2.0.6
...
build(deps): bump github.com/hashicorp/golang-lru/v2 from 2.0.4 to 2.0.6
2023-09-05 21:05:45 +00:00
Michael Eischer
df53f4782b
Merge pull request #4453 from lbausch/docs/fix-table
...
docs: fix malformed table
2023-09-05 20:37:30 +00:00
Michael Eischer
35055adbc4
Merge pull request #4454 from lbausch/tweak-readme
...
README: add Mastodon link
2023-09-05 20:36:32 +00:00
dependabot[bot]
bd9eb528c0
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from a9794064588be971151ec5e7144cb535bcb56e36 to cf8514a65188af1d4f94f8c28a7a4153af1088ce.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](a979406458...cf8514a651
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-09-01 01:45:03 +00:00
dependabot[bot]
02032f3109
build(deps): bump github.com/hashicorp/golang-lru/v2 from 2.0.4 to 2.0.6
...
Bumps [github.com/hashicorp/golang-lru/v2](https://github.com/hashicorp/golang-lru ) from 2.0.4 to 2.0.6.
- [Release notes](https://github.com/hashicorp/golang-lru/releases )
- [Commits](https://github.com/hashicorp/golang-lru/compare/v2.0.4...v2.0.6 )
---
updated-dependencies:
- dependency-name: github.com/hashicorp/golang-lru/v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-09-01 01:41:13 +00:00
dependabot[bot]
cfff1367c1
build(deps): bump google.golang.org/api from 0.134.0 to 0.138.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.134.0 to 0.138.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.134.0...v0.138.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-09-01 01:40:56 +00:00
Lorenz Bausch
353265a329
README: add Mastodon link
2023-08-27 10:52:12 +02:00
Lorenz Bausch
84a1170dee
docs: remove trailing whitespace
2023-08-27 10:49:34 +02:00
Lorenz Bausch
0b4d9c9a51
docs: add missing whitespace to avoid malformed table warning
2023-08-27 10:46:55 +02:00
Lorenz Bausch
5422a7daa5
docs: explain meaning of asterisk when managing repository keys
2023-08-27 10:40:55 +02:00
greatroar
691c01963b
internal/restic: Return summary from BlobSet.String
...
Fixes #4449 : error messages from prune were too long to fit in scroll
buffers.
2023-08-25 21:41:30 +02:00
Michael Eischer
2bec99dc6f
master_index: fix inconsistent length blob length in test
...
Two blobs with the same hash must always have the same content length.
2023-08-19 20:04:25 +02:00
Michael Eischer
e60c5b2d7f
Merge pull request #4436 from MichaelEischer/add-snapshot-subfolder-syntax-to-help-text
...
Add snapshot subfolder syntax in help output of diff/dump/restore
2023-08-13 19:36:38 +02:00
Michael Eischer
a04964bb86
Merge pull request #4446 from 2000yeshu/feat-cat
...
Check for arguments before opening remote connection in cat command
2023-08-13 17:25:37 +00:00
Michael Eischer
fe54912a46
cat: extract parameter validation and add a test
2023-08-13 19:17:46 +02:00
Michael Eischer
feb6abb7bb
Merge pull request #4394 from greatroar/acl
...
dump: Rewrite Linux ACL handling
2023-08-13 17:01:31 +00:00
greatroar
aaf5254e26
dump: Rewrite Linux ACL handling
...
The old version was taken from an MPL-licensed library. This is a
cleanroom implementation. The code is shorter and it's now explicit that
only Linux ACLs are supported.
2023-08-13 13:37:11 +02:00
Yakul Garg
705556f134
Check for arguments before opening remote connection in cat command
...
Signed-off-by: Yakul Garg <2000yeshu@gmail.com>
2023-08-12 23:54:40 +05:30
Michael Eischer
c23eebc225
Merge pull request #4428 from MichaelEischer/fix-restore-help
...
restore: Fix help text for `--iinclude` and `--iexclude`
2023-08-12 20:21:22 +02:00
Refutable4890
e09f6f540f
Add a description of the resumption of `copy`
2023-08-08 12:51:40 +08:00
Michael Eischer
0fca028491
restore: Fix help text for `--iinclude` and `--iexclude`
2023-08-05 18:21:03 +02:00
Michael Eischer
57a08291f5
Merge pull request #4429 from restic/dependabot/github_actions/docker/login-action-a9794064588be971151ec5e7144cb535bcb56e36
...
build(deps): bump docker/login-action from 65b78e6e13532edd9afa3aa52ac7964289d1a9c1 to a9794064588be971151ec5e7144cb535bcb56e36
2023-08-05 13:41:08 +00:00
Michael Eischer
2117ce4cfb
Merge pull request #4434 from lbausch/docs-fix-wording
...
docs: fix `percent_done` description
2023-08-05 13:39:02 +00:00
Michael Eischer
3a478bc522
unify snapshotID spelling in command parameters
2023-08-05 12:35:45 +02:00
Michael Eischer
9a7704fa2b
diff/dump/restore: describe snapshot:subfolder syntax in help text
2023-08-05 12:35:08 +02:00
Lorenz Bausch
9a69f44de2
docs: fix `percent_done` description
2023-08-04 16:15:18 +02:00
dependabot[bot]
7a6339180b
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from 65b78e6e13532edd9afa3aa52ac7964289d1a9c1 to a9794064588be971151ec5e7144cb535bcb56e36.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](65b78e6e13...a979406458
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-08-01 01:19:51 +00:00
Alexander Neumann
82e6e28781
Set development version for 0.16.0
2023-07-31 20:24:29 +02:00
Alexander Neumann
eb33e564c9
Add version for 0.16.0
2023-07-31 20:23:24 +02:00
Alexander Neumann
62680af734
Update manpages and auto-completion
2023-07-31 20:23:24 +02:00
Alexander Neumann
68460fd3d1
Generate CHANGELOG.md for 0.16.0
2023-07-31 20:22:50 +02:00
Alexander Neumann
9459328d8d
Prepare changelog for 0.16.0
2023-07-31 20:22:50 +02:00
rawtaz
9d71990c26
Merge pull request #4424 from MichaelEischer/tweak-changelog
...
Tweak changelog entries
2023-07-29 11:29:30 +00:00
rawtaz
9219b6a0ef
Merge pull request #4427 from MichaelEischer/changelog-template
...
Extend style guide in changelog template
2023-07-29 11:15:17 +00:00
Michael Eischer
9a0ec05145
Extend style guide in changelog template
2023-07-29 12:55:56 +02:00
Michael Eischer
2a5b6af2e9
extend gc tweak changelog
2023-07-29 12:51:05 +02:00
Leo R. Lundgren
51e4e5ef82
doc: Polish changelog
2023-07-29 12:51:04 +02:00
Michael Eischer
e998314088
tweak changelog entries
2023-07-29 12:50:01 +02:00
Michael Eischer
be03c1a457
Merge pull request #4422 from MichaelEischer/document-non-utf-symlinks
...
Document `linktarget_raw` field for non utf symlinks
2023-07-28 23:14:08 +02:00
Michael Eischer
95050117eb
Merge pull request #4426 from MichaelEischer/consistent-forget-policy
...
forget: replace `--keep-* -1` with `--keep-* unlimited`
2023-07-28 19:06:00 +00:00
Michael Eischer
21a3a41b69
Merge pull request #4423 from MichaelEischer/update-dependencies
...
Update dependencies
2023-07-28 19:25:02 +02:00
Michael Eischer
3f919f2371
forget: simplify test
2023-07-28 19:21:40 +02:00
Michael Eischer
50b43fbac0
forget: replace `--keep-* -1` with `--keep-* unlimited`
...
This ensures consistency with the `prune --max-unused unlimited` option.
2023-07-28 19:21:40 +02:00
Michael Eischer
f689e2638d
reenable aix builds
2023-07-27 21:27:31 +02:00
Michael Eischer
f9ef2b1e44
update all dependencies
2023-07-27 21:26:44 +02:00
Michael Eischer
b87a37f318
doc: Fix typos
2023-07-25 20:14:56 +02:00
Michael Eischer
647ebf352a
doc: Describe linktarget_raw field
2023-07-25 20:14:47 +02:00
Michael Eischer
62d3ef4a93
Merge pull request #4417 from MichaelEischer/fix-flaky-list-cancel-test
...
backend: Fix unreliable TestListCancel/Timeout test
2023-07-23 18:04:11 +02:00
Michael Eischer
6c5fc32967
Merge pull request #3225 from wojas/restic-cmd-more-help
...
Extend help output with short overview
2023-07-23 15:43:03 +00:00
Michael Eischer
ae13cf15c1
doc: move quickstart to docs
2023-07-23 17:32:37 +02:00
Michael Eischer
1fe1ec40a2
Merge pull request #4409 from arjunajesh/issue#1495
...
snapshots sorted by timestamp in restic find
2023-07-23 11:00:41 +00:00
arjunajesh
32f5ee6f4e
snapshots sorted by timestamp
2023-07-23 12:49:23 +02:00
Konrad Wojas
5bd8a6d7eb
Extend help output with short overview
...
The list of subcommands can be confusing for first time users. This
adds a short overview of the most important commands to help get people
started.
2023-07-23 12:01:40 +02:00
Michael Eischer
4a33370072
backend: Fix unreliable TestListCancel/Timeout test
...
The test uses `WithTimeout` to create a context that cancels the List
operation after a given delay. Several backends internally use a derived
child context created using WithCancel.
The cancellation of a context first closes the done channel of the
context (here: the `WithTimeout` context) and _afterwards_ propagates
the cancellation to child contexts (here: the `WithCancel` context).
Therefor if the List implementation uses a child context, then it may
take a moment until that context is also cancelled. Thus give the
context cancellation a moment to propagate.
2023-07-23 11:47:56 +02:00
Michael Eischer
98fb56baa6
Merge pull request #3802 from MichaelEischer/support-non-utf-symlinks
...
Support non utf symlink targets
2023-07-23 00:15:01 +02:00
Michael Eischer
f12bbd9229
restic: check that Node.LinkTarget can handle non-utf8 characters
2023-07-23 00:05:14 +02:00
Michael Eischer
6adb629608
Add support for non-utf8 symlink targets
2023-07-23 00:05:14 +02:00
Michael Eischer
25ff9fa893
Merge pull request #4334 from MichaelEischer/snapshot-subtree-syntax
...
Add support for snapshot:path syntax
2023-07-22 23:59:26 +02:00
Michael Eischer
bdaec8fdb8
Merge pull request #4374 from MichaelEischer/try-refresh-stale-locks
...
Try to refresh stale locks
2023-07-22 23:50:52 +02:00
Michael Eischer
55b440b520
tweak snapshot:subfolder docs
2023-07-22 23:50:17 +02:00
Michael Eischer
3f63b53090
lock: Shrink critical section in RefreshStaleLock
...
A cleaning up after a failed lock refresh attempt does not require a
mutex.
2023-07-22 23:29:14 +02:00
Michael Eischer
76253b2a20
Hopefully fix flaky TestLockWaitCancel test
2023-07-22 23:23:17 +02:00
Michael Eischer
20e82d1fcf
lock: add changelog for stale lock refresh
2023-07-22 23:23:17 +02:00
Michael Eischer
96eada3d5f
backend/sema: add test for freeze functionality
2023-07-22 23:23:17 +02:00
Michael Eischer
b8f4267a36
lock: add more debug information to flaky test
2023-07-22 23:23:17 +02:00
Michael Eischer
1f6883a05c
lock: test stale lock refresh
2023-07-22 23:23:17 +02:00
Michael Eischer
8154f6a77a
Ensure consistent naming for `<snapshot>:<subfolder>` syntax
2023-07-22 19:55:57 +02:00
Michael Eischer
184b7616ba
document snapshot:subfolder syntax
2023-07-22 19:55:57 +02:00
Michael Eischer
67f237b4f3
add changelog for snapshot:path syntax
2023-07-22 19:55:57 +02:00
Michael Eischer
090f9d6237
restic: Cleanup and simplify TestCreateSnapshot
2023-07-22 19:55:57 +02:00
Michael Eischer
321cc35cde
restic: add test for FindTreeDirectory
2023-07-22 19:55:57 +02:00
Michael Eischer
a97915642c
restic: Ensure snapshots created by TestCreateSnapshot are valid
...
All nodes in a tree must be sorted by node name. Otherwise functionality
like searching for a specific tree node will break.
2023-07-22 19:55:57 +02:00
Michael Eischer
c64d81063e
test snapshot filtering with snapshot:path syntax
2023-07-22 19:55:57 +02:00
Michael Eischer
233b841ad9
Add cat tree snapshot:subpath command
...
This command prints the tree for the given path.
2023-07-22 19:55:57 +02:00
Michael Eischer
85860e6e97
Add support for snapshot:subpath syntax
...
This snapshot specification syntax is supported by the cat, diff, dump,
ls and restore command.
2023-07-22 19:55:57 +02:00
Michael Eischer
2dd6769429
lock: Fix possible deadlock during refresh of stale lock
...
A delayed lock refresh could send a signal on the `refreshed` channel
while the `monitorLockRefresh` goroutine waits for a reply to its
`refreshLockRequest`. As the channels are unbuffered, this resulted in a
deadlock.
2023-07-22 19:55:44 +02:00
Michael Eischer
5d9b0d894e
lock: add unit test for RefreshStaleLock
2023-07-22 19:55:44 +02:00
Michael Eischer
d4bf7a3cb1
restic: cleanup lock test
2023-07-22 19:55:44 +02:00
Michael Eischer
24c8a33da9
restic: reduce sleeps in lock tests
2023-07-22 19:55:44 +02:00
Michael Eischer
399f8e84a1
lock: use mem backend for locking tests
...
This hopefully fixes the timing related test failures which appear to be
caused by very slow fsync calls.
2023-07-22 19:55:44 +02:00
Michael Eischer
6a436d731d
lock: rename test helper function
2023-07-22 19:55:44 +02:00
Michael Eischer
b2fcbc21cb
lock: rework stale lock refresh to avoid data race
2023-07-22 19:55:44 +02:00
Michael Eischer
05e5e29a8c
backup: don't pretend to lock repo for dry run
...
The dry run did not actually create lock files, but still ran all other
parts of the lock processing.
2023-07-22 19:55:44 +02:00
Michael Eischer
f490288738
lock: freeze backend operations while refreshing stale lock
...
Freeze new backend operations while trying to refresh a stale lock.
2023-07-22 19:55:44 +02:00
Michael Eischer
51718ec561
restic: generic implemention of AsBackend
2023-07-22 19:55:44 +02:00
Michael Eischer
11eb88a2ea
lock: cleanup
2023-07-22 19:55:44 +02:00
Michael Eischer
bee3231ed4
lock: try refreshing of stale locks
...
A stale lock may be refreshed if it continues to exist until after a
replacement lock has been created. This ensures that a repository was
not unlocked in the meantime.
2023-07-22 19:55:44 +02:00
Michael Eischer
60d8066568
Merge pull request #4407 from MichaelEischer/simpler-required-ci
...
CI: Add check that verifies the results of other checks
2023-07-22 19:55:06 +02:00
Michael Eischer
08dea911bd
CI: Add check that verifies the results of other checks
...
To properly use merge queues, it is necessary to mark certain checks as
"required" in the branch protection rules. However, this doesn't scale
with matrix jobs and would require manual tweaks of the CI jobs change.
The new analyze job simplifies this by allowing the branch protection
rule just check for that job. All further dependencies are then
configured within the CI config.
2023-07-16 14:13:41 +02:00
Michael Eischer
47206a6579
Merge pull request #4351 from kenny-y-dev/human-readable-ls
...
Add --human-readable to ls and find output
2023-07-16 11:44:41 +00:00
Michael Eischer
594689db32
Merge pull request #4400 from MichaelEischer/rest-handle-missing-dirs
...
rest: Ignore missing directories
2023-07-14 23:29:37 +02:00
Michael Eischer
5705326bb8
add changelog
2023-07-14 22:39:38 +02:00
Michael Eischer
978ebaac49
rest: use http status code constants
2023-07-14 22:39:38 +02:00
Michael Eischer
307aeb6849
rest: Don't return error if listing non-existent directory
...
When transferring a repository from S3 to, for example, a local disk
then all empty folders will be missing.
When saving files, the missing intermediate folders are created
automatically. Therefore, missing directories can be ignored by the
`List()` operation.
2023-07-14 22:39:38 +02:00
Michael Eischer
9cd85d5956
Merge pull request #4029 from ekarlso/add-azure-workload-identity-support
...
add azure workload identity support
2023-07-09 19:06:59 +00:00
Michael Eischer
c9f506925c
azure: Document additional auth options
2023-07-09 20:56:21 +02:00
Michael Eischer
e990d3d483
azure: Support authentication using managed / workload identity
...
See https://github.com/Azure/azure-sdk-for-go/tree/sdk/azidentity/v1.3.0/sdk/azidentity
2023-07-09 14:47:03 +02:00
Michael Eischer
7042190807
Merge pull request #4403 from MichaelEischer/fix-certificate-env-handling
...
Fix certificate env handling
2023-07-09 14:46:38 +02:00
Michael Eischer
c158741e2e
CI: add minimal CLI test
...
Just create a repository and run a minimal backup.
2023-07-09 14:29:43 +02:00
Michael Eischer
89fbd39e59
Don't print stacktrace on invalid cacert option
2023-07-09 14:23:54 +02:00
Michael Eischer
1ce599d2ae
Fix handling of empty cacert environment variable
...
This resulted in a "empty filename for root certificate supplied" error.
2023-07-09 14:23:54 +02:00
Michael Eischer
789fec3da7
Merge pull request #4402 from MichaelEischer/merge-queue-setup
...
CI: Enable missing CI tests for Github merge queue
2023-07-09 13:57:10 +02:00
Michael Eischer
8a120c8800
CI: Enable missing CI tests for Github merge queue
2023-07-08 23:51:39 +02:00
Michael Eischer
41395e83c5
Merge pull request #2740 from metalsp0rk/document-json
...
Add JSON documentation
2023-07-08 21:34:02 +00:00
Michael Eischer
2a193195b0
Merge pull request #4364 from MichaelEischer/multiplatform-docker
...
Setup multiplatform docker builds
2023-07-08 23:31:58 +02:00
Michael Eischer
229c7b24a4
doc: add program_version field of snapshot JSON output
2023-07-08 23:31:20 +02:00
Michael Eischer
b34ce57dd4
doc: describe JSON output of restore command
2023-07-08 23:31:20 +02:00
Michael Eischer
1ce839228e
doc: Expand JSON documentation
2023-07-08 23:31:20 +02:00
Kyle Brennan
fb1170c1d6
doc tweaks
2023-07-08 23:21:41 +02:00
Kyle Brennan
e457fe22bc
Documentation improvement
2023-07-08 23:21:41 +02:00
Kyle Brennan
39299e36ef
add json documentation
2023-07-08 23:21:41 +02:00
Michael Eischer
0512f292e8
Merge pull request #4361 from MichaelEischer/stats-repo-debug-info
...
stats: Add `debug` mode to collect repository statistics
2023-07-08 23:20:23 +02:00
Michael Eischer
3a93e28605
CI: Remove .dockerignore to ensure reproducible builds
...
Since go 1.18, built binaries also include VCS information such as the
built commit. This information is also included in the official
binaries. To ensure that the Docker container recreates the same
binaries, the .git folder must also be transferred into the container.
Thus, remove the .dockerignore file.
The copied files must also be owned by the current user within the
container, as git refuses to work otherwise.
2023-07-08 23:16:14 +02:00
Michael Eischer
6b82cce1bd
add changelog for multiplatform containers
2023-07-08 23:16:14 +02:00
Michael Eischer
31e07cecbb
docker: update to Go 1.20 for custom container builds
2023-07-08 23:05:39 +02:00
Michael Eischer
c181b51360
doc: Update docker build process
2023-07-08 23:05:38 +02:00
Michael Eischer
ccd19b7e88
CI: run cloud backend tests only on restic/restic
2023-07-08 23:05:02 +02:00
Michael Eischer
b0987ff570
CI: only build containers on restic/restic
2023-07-08 23:05:02 +02:00
Michael Eischer
eff3124f15
CI: Setup automatic container builds for ghcr.io
...
Containers are built for new tags and pushes to the master branch.
2023-07-08 23:05:02 +02:00
Michael Eischer
43fa051546
Directly build restic binary in release Docker container
2023-07-08 23:05:02 +02:00
Michael Eischer
2293835242
Release multi-platform docker containers
2023-07-08 23:05:02 +02:00
Michael Eischer
0fcb6c7f94
ask for debug statistics in resource usage issues
2023-07-08 23:04:47 +02:00
Michael Eischer
325fa916b5
stats: Add debug mode to collect repository statistics
2023-07-08 23:04:47 +02:00
Michael Eischer
170e495334
Merge pull request #4401 from MichaelEischer/fix-windows-build
...
fuse: fix windows build
2023-07-08 23:01:22 +02:00
Michael Eischer
9d44682e3e
fuse: fix windows build
2023-07-08 22:40:22 +02:00
Michael Eischer
4d43509423
Merge pull request #4379 from chenxiaolong/symlink_xattrs
...
Add support for extended attributes on symlinks
2023-07-08 16:56:51 +00:00
Michael Eischer
ea9ad77e05
restic: refactor node test
2023-07-08 18:54:56 +02:00
Michael Eischer
cc84884d2e
restic: basic xattr test for files/dirs
2023-07-08 18:49:21 +02:00
Michael Eischer
4a5ae2ba84
restic: test NodeFromFileInfo for symlinks
2023-07-08 18:18:13 +02:00
Michael Eischer
1f1e50f49e
fuse: add test for symlink xattr
2023-07-08 18:02:17 +02:00
Michael Eischer
f3c3b0f377
fuse: deduplicate xattr code
2023-07-08 17:41:45 +02:00
Michael Eischer
7e2be9e081
Merge pull request #4384 from arjunajesh/issue#1926
...
certificates can be passed through env vars
2023-07-08 07:46:31 +00:00
Michael Eischer
cc3c218baf
small cleanups for certificate environment variables
2023-07-08 09:44:59 +02:00
arjunajesh
c1578a2035
certificates can be passed through env vars
2023-07-08 09:44:59 +02:00
Michael Eischer
678b983300
Merge pull request #4360 from Refutable4890/master
...
Add an description of `check` temporary cache directory location
2023-07-07 21:51:52 +00:00
Michael Eischer
1b3870dc43
Merge pull request #4378 from MichaelEischer/add-version-to-snapshot
...
Add program version to snapshot
2023-07-07 23:50:44 +02:00
Michael Eischer
ef40aee2bd
Merge pull request #4373 from MichaelEischer/performance-issue-help
...
document how to analyze performance / memory usage issues
2023-07-07 23:50:27 +02:00
Michael Eischer
3a32c4e59f
document how to analyze performance / memory usage issues
2023-07-07 23:38:53 +02:00
Michael Eischer
e703e89e9b
add changelog for program version
2023-07-07 23:36:57 +02:00
Michael Eischer
389f6ee74c
backup: add minimal test for program versioni
2023-07-07 23:33:23 +02:00
Michael Eischer
bbac74b172
add program version to snapshot
2023-07-07 23:27:10 +02:00
Michael Eischer
a280b7364e
Merge pull request #4395 from greatroar/parsebytes
...
cmd, ui: Move size parsing code and make it more robust
2023-07-07 21:24:47 +00:00
Michael Eischer
825651a135
Merge pull request #4387 from arjunajesh/issue#2468
...
azure domain can be overridden with env variable
2023-07-07 21:17:01 +00:00
Michael Eischer
e36d17a6f8
azure: tweak documentation for endpoint suffix
2023-07-07 23:10:26 +02:00
arjunajesh
068b115abc
added azure domain parameter
2023-07-07 23:10:26 +02:00
greatroar
41a5bf357f
cmd, ui: Move size parsing code and make it more robust
2023-07-02 21:44:50 +02:00
Michael Eischer
f96896a9c0
Merge pull request #4383 from LouisMT/add-docker-hostname
...
Add hostname flag to Docker docs
2023-07-01 18:58:43 +02:00
dependabot[bot]
2ab9a3b9c3
Merge pull request #4389 from restic/dependabot/go_modules/google.golang.org/api-0.129.0
2023-07-01 16:57:18 +00:00
Louis Matthijssen
dbe2eef80c
Add hostname flag to Docker docs
...
Fixes #4380
2023-07-01 18:46:31 +02:00
dependabot[bot]
6e7c6674ad
build(deps): bump google.golang.org/api from 0.116.0 to 0.129.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.116.0 to 0.129.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.116.0...v0.129.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-01 16:11:44 +00:00
Michael Eischer
7fe830ee1e
Merge pull request #4392 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.6.1
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.6.0 to 1.6.1
2023-07-01 18:10:47 +02:00
Michael Eischer
a07c7166ba
Merge pull request #4390 from restic/dependabot/go_modules/golang.org/x/oauth2-0.9.0
...
build(deps): bump golang.org/x/oauth2 from 0.8.0 to 0.9.0
2023-07-01 18:10:38 +02:00
dependabot[bot]
d2f8f9de23
build(deps): bump golang.org/x/oauth2 from 0.8.0 to 0.9.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.8.0 to 0.9.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.8.0...v0.9.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-01 15:32:55 +00:00
dependabot[bot]
d5fe5107c8
Merge pull request #4391 from restic/dependabot/go_modules/golang.org/x/sys-0.9.0
2023-07-01 15:31:42 +00:00
dependabot[bot]
f08ba1a005
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.6.0 to 1.6.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.6.0...sdk/azcore/v1.6.1 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-01 01:23:19 +00:00
dependabot[bot]
70fb554854
build(deps): bump golang.org/x/sys from 0.8.0 to 0.9.0
...
Bumps [golang.org/x/sys](https://github.com/golang/sys ) from 0.8.0 to 0.9.0.
- [Commits](https://github.com/golang/sys/compare/v0.8.0...v0.9.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sys
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-01 01:23:05 +00:00
Andrew Gunnerson
8c02ebb029
Add support for extended attributes on symlinks
...
Linux allows the use of non-`user.` extended attributes on symlinks. One
of the main users of this functionality is SELinux's `security.selinux`
xattr for storing a path's label. By storing symlink xattrs, restic is
now suitable for backing up the root filesystem on Linux distributions
that use SELinux.
This commit adds support for symlink xattrs when backing up data,
restoring data, and mounting snapshots via a fuse mount. All calls to
the xattr library have been updated to the use `L` variants of the
various functions, which always operate on the path given, without
following symlinks.
Fixes : #4375
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-06-19 14:37:31 -04:00
Michael Eischer
8dcb0c4a9d
doc: improve description of caching behavior of the check command
2023-06-17 22:18:50 +02:00
Michael Eischer
74ca82a6f8
Merge pull request #4362 from restic/unified-backend-factory
...
Unified backend initialization
2023-06-17 15:52:37 +02:00
Michael Eischer
9cbc2502c6
Merge pull request #4366 from gautammenghani/master
...
Issue #3624 : Preserve oldest snapshot when keep-* values are not sati…
2023-06-17 15:41:17 +02:00
Michael Eischer
93038ed8f4
s3: restore retries for minio tests
2023-06-17 15:25:08 +02:00
Gautam Menghani
8da5a6649b
Preserve oldest snapshot when keep-within* does not collect enough
2023-06-17 15:17:20 +02:00
Michael Eischer
3888c21a27
reword changelog
2023-06-17 15:16:58 +02:00
Michael Eischer
1257c2c075
forget: Add comments to snapshot policy
2023-06-17 15:16:58 +02:00
Gautam Menghani
182b9796e4
Issue #3624 : Preserve oldest snapshot when keep-* values are not satisfied
2023-06-17 15:16:58 +02:00
Michael Eischer
cbf87fbdb3
init: don't include password in debug log
2023-06-17 15:15:58 +02:00
Michael Eischer
b5511e8e4c
Fix linter warnings
2023-06-17 15:15:58 +02:00
Michael Eischer
50e0d5e6b5
backend: Hardcode backend scheme in Factory
...
Our ParseConfig implementations always expect a specific scheme, thus no
other scheme would work.
2023-06-17 15:15:58 +02:00
Michael Eischer
705ad51bcc
backend: check that StripPassword can be called
2023-06-17 15:15:58 +02:00
Michael Eischer
13a8b5822f
backend: Adjust tests to use the Factory to instantiate the backend
...
This drastically reduces the amount of duplicated test code.
2023-06-17 15:15:58 +02:00
Michael Eischer
3d3bb88745
b2: remove duplicate check for config file during repository creation
...
No other backend implements that check. The check that a repository is
not yet initialized is handled by the Repository later on.
2023-06-17 15:15:58 +02:00
Michael Eischer
3a3cf608f5
b2/s3: Move config validation from ApplyEnvironment to Open/Create
...
Conceptually the backend configuration should be validated when creating
or opening the backend, but not when filling in information from
environment variables into the configuration.
2023-06-17 15:15:58 +02:00
Michael Eischer
19ac12d95b
location: make genericBackendFactory private
2023-06-17 15:15:58 +02:00
Michael Eischer
3325a7c862
location: extract backend specific part of StripPassword
...
The tests for the rest backend now reside there.
2023-06-17 15:15:58 +02:00
Michael Eischer
555be49a79
location: Make ParseConfig-test backend agnostic
...
The backend specific parts of the test are now directly handled by the
respective backend. Duplicate tests were removed.
2023-06-17 15:15:58 +02:00
Michael Eischer
9aa9e0d1ec
local/sftp: move limiter setup into backend
2023-06-17 15:15:58 +02:00
Michael Eischer
7d12c29286
backend: Unify backend construction using factory and registry
...
This unified construction removes most backend-specific code from
global.go. The backend registry will also enable integration tests to
use custom backends if necessary.
2023-06-17 15:15:57 +02:00
Michael Eischer
56836364a4
backend: pass context into every backend constructor
2023-06-17 15:15:57 +02:00
Michael Eischer
4df77e9f26
Merge pull request #4314 from MichaelEischer/restore-json
...
Add JSON support for the restore progress bar
2023-06-16 23:53:17 +02:00
Michael Eischer
2545c84321
Merge pull request #4365 from MichaelEischer/ci-rework-build-tests
...
CI: rework build tests
2023-06-16 23:29:39 +02:00
Michael Eischer
191c47d30e
Merge pull request #4353 from MichaelEischer/tune-gc
...
Tune Go garbage collector
2023-06-16 23:24:39 +02:00
Michael Eischer
dd1ef13c1c
Merge pull request #4352 from MichaelEischer/pointerless-index
...
index: optimize go GC performance
2023-06-16 23:23:14 +02:00
Michael Eischer
4d5ee987a7
add changelog about missing AIX builds
2023-06-16 23:19:36 +02:00
Michael Eischer
b2ed42cec4
index: add basic hat test
2023-06-16 23:12:30 +02:00
Michael Eischer
61042a77a4
building on aix is currently not possible
2023-06-09 13:35:35 +02:00
Michael Eischer
4e9e2c3229
CI: Use build-release-binaries to run the cross-compilation tests
...
gox silently ignored linux/mips and aix/ppc64. This change also removes
the duplicate platform list.
2023-06-09 13:27:57 +02:00
Michael Eischer
faec0ff816
build-release-binaries: support building a subset of all platforms
2023-06-09 13:27:33 +02:00
Michael Eischer
07d1f8047e
restore: More descriptive field names for the JSON output
2023-06-09 12:07:55 +02:00
Michael Eischer
b2b0760eb0
restore: add test for json output
2023-06-09 12:03:33 +02:00
Michael Eischer
cf16239058
restore: print output via termStatus
2023-06-09 12:03:33 +02:00
Michael Eischer
1531eab746
mention restore json support in changelog
2023-06-09 12:03:33 +02:00
Michael Eischer
d54176ce5d
restore: add basic json progress
2023-06-09 12:03:33 +02:00
Michael Eischer
a9aff885d6
restore: reorganize progress bar code
...
The structure is now much more similar to that of the backup command.
2023-06-09 12:03:33 +02:00
Michael Eischer
bb20078641
restore: pass termStatus to restore in tests
2023-06-09 12:03:33 +02:00
Michael Eischer
237f32c651
Merge pull request #3261 from DRON-666/prealloc-packs
...
Prevent local backend file fragmentation by file preallocation.
2023-06-09 11:50:46 +02:00
DRON-666
74e4656850
Update changelog
2023-06-09 11:49:06 +03:00
DRON-666
c37d587f81
Use PreallocateFile in local backend
2023-06-09 11:49:06 +03:00
DRON-666
ffc6b3d887
Move PreallocateFile to fs package
2023-06-09 11:49:06 +03:00
Michael Eischer
88c63a029c
Merge pull request #3563 from matta/improve-eta
...
Improve the ETA displayed during backup
2023-06-08 20:49:19 +02:00
Michael Eischer
0b908bb1fb
Address review comments
2023-06-08 20:24:21 +02:00
Matt Armstrong
0372c7ef04
Improve the ETA displayed during backup
...
The ETA restic displays was based on a rate computed across the entire
backup operation. Often restic can progress at uneven rates. In the worst
case, restic progresses over most of the backup at a very high rate and
then finds new data to back up. The displayed ETA is then unrealistic and
never adapts.
Restic now estimates the transfer rate based on a sliding window, with the
goal of adapting to observed changes in rate. To avoid wild changes in the
estimate, several heuristics are used to keep the sliding window wide
enough to be relatively stable.
2023-06-08 20:05:56 +02:00
Michael Eischer
9464c63550
Make formatNode test timezone independent
...
formatNode formats the timestamp according to the current time zone. Pin
the local timezone to UTC to ensure the test works everywhere.
2023-06-08 19:18:30 +02:00
Michael Eischer
6ebf2dd235
Reduce duplicate code in test for fomatNode
2023-06-08 19:16:16 +02:00
Michael Eischer
5f153109ba
Refactor formatNode
2023-06-08 19:12:49 +02:00
Michael Eischer
2beaa74892
tweak changelog
2023-06-08 19:12:26 +02:00
Michael Eischer
55c21846b1
Revert "index: remove redundant storage of indexmap size"
...
This reverts commit f1c388c623
.
For an uninitialized indexmap the returned size was `-1` which is
unexpected and could cause problems.
2023-06-08 18:08:46 +02:00
Michael Eischer
0f80b6a137
add changelog for gc tuning
2023-06-08 18:02:46 +02:00
Michael Eischer
e14ccb1142
Merge pull request #4298 from restic/backend-parseconfig-cleanup
...
Unified and slightly type-safer backend config parsing
2023-06-08 12:02:27 +02:00
Michael Eischer
609367195a
backend: Fix test compilation with Go 1.18 and 1.19
2023-06-07 22:31:15 +02:00
Michael Eischer
18eb1d3ab0
backend: test cleanup
2023-06-07 22:31:15 +02:00
Michael Eischer
32a6b66267
backend: add standardized Config.ApplyEnvironment
...
This removes the backend specific special cases while parsing the
configuration in `global.go`.
2023-06-07 22:31:15 +02:00
Michael Eischer
f903db492c
backend: let ParseConfig return a Config pointer
...
In order to change the backend initialization in `global.go` to be able
to generically call cfg.ApplyEnvironment() for supported backends, the
`interface{}` returned by `ParseConfig` must contain a pointer to the
configuration.
An alternative would be to use reflection to convert the type from
`interface{}(Config)` to `interface{}(*Config)` (from value to pointer
type). However, this would just complicate the type mess further.
2023-06-07 22:31:15 +02:00
Michael Eischer
25a0be7f26
backend: fix linter warnings
2023-06-07 22:31:15 +02:00
Michael Eischer
a27b7f1370
backend: use generic instead of any type for test suite
2023-06-07 22:31:14 +02:00
Michael Eischer
fa361dbfbd
backend: use generic implementation for ParseConfig tests
2023-06-07 22:30:35 +02:00
Michael Eischer
5260d38980
backend: let ParseConfig return concrete type
2023-06-07 22:30:35 +02:00
Michael Eischer
2f7b4ceae1
backend: Move environment based configuration into backend
2023-06-07 22:30:35 +02:00
Refutable4890
aea7538936
Add an description of `check` temporary cache directory location
2023-06-07 22:35:33 +08:00
Michael Eischer
49a6a4f5bf
Merge pull request #4356 from MichaelEischer/fix-unlock-hint
...
lock: fix missing hint to unlock command if repository is locked
2023-06-06 21:03:06 +02:00
Michael Eischer
237d00000e
lock: fix missing hint to unlock command if repository is locked
2023-06-02 23:16:49 +02:00
dependabot[bot]
0c727f6ad1
Merge pull request #4355 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.56
2023-06-02 20:51:26 +00:00
Michael Eischer
4e7d3efad4
Merge pull request #4345 from DanielGibson/patch-1
...
restic restore docs: Mention that restic restore is fastest
2023-06-02 22:44:38 +02:00
Michael Eischer
17446da5fd
Merge pull request #4343 from greatroar/cache
...
cache: Restructure New to remove redundant operations
2023-06-02 22:41:17 +02:00
dependabot[bot]
a3cee840d2
build(deps): bump github.com/minio/minio-go/v7 from 7.0.52 to 7.0.56
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.52 to 7.0.56.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.52...v7.0.56 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 20:35:42 +00:00
dependabot[bot]
c76f3a1e27
Merge pull request #4348 from restic/dependabot/go_modules/github.com/spf13/cobra-1.7.0
2023-06-02 20:34:38 +00:00
dependabot[bot]
1e9714088d
Merge pull request #4347 from restic/dependabot/go_modules/golang.org/x/sync-0.2.0
2023-06-02 20:32:27 +00:00
Michael Eischer
58e3f5955c
Merge pull request #4350 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.6.0
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.4.0 to 1.6.0
2023-06-02 22:26:55 +02:00
dependabot[bot]
dd8d8b1ae0
build(deps): bump github.com/spf13/cobra from 1.6.1 to 1.7.0
...
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra ) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/spf13/cobra/releases )
- [Commits](https://github.com/spf13/cobra/compare/v1.6.1...v1.7.0 )
---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 20:22:39 +00:00
dependabot[bot]
e1ac0f0e0c
build(deps): bump golang.org/x/sync from 0.1.0 to 0.2.0
...
Bumps [golang.org/x/sync](https://github.com/golang/sync ) from 0.1.0 to 0.2.0.
- [Commits](https://github.com/golang/sync/compare/v0.1.0...v0.2.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sync
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 20:22:36 +00:00
Michael Eischer
a88d90b8e5
Merge pull request #4346 from restic/dependabot/go_modules/golang.org/x/oauth2-0.8.0
...
build(deps): bump golang.org/x/oauth2 from 0.7.0 to 0.8.0
2023-06-02 22:21:22 +02:00
Michael Eischer
2fcb3947df
prune: trigger GC after prune planning
2023-06-02 21:57:40 +02:00
Michael Eischer
eef0ee7a85
repository: trigger GC after loading the index
...
Loading the index requires some scratch space, thus make sure that this
memory does not factor into the targeted gc memory usage limit.
2023-06-02 21:56:14 +02:00
Michael Eischer
f1b73c9301
Reduce GOGC to 50
...
The index used by restic consumes a major part of the total memory
usage. This leads to an unnecessarily large amount of memory that
contains ephemeral objects that are only used for a short time.
2023-06-02 21:51:50 +02:00
Michael Eischer
ac1dfc99bb
index: fix blocklist size
2023-06-02 19:39:12 +02:00
Kenny Y
098de3554c
Add pull request link
2023-06-01 21:30:51 -04:00
Kenny Y
8812dcd56a
Add --human-readable to ls and find output
...
Modifies format module to add options for human readable storage size formatting, using size parsing already in ui/format.
Cmd flag --human-readable added to ls and find commands.
Additional option added to formatNode to support printing size in regular or new human readable format
2023-06-01 21:18:18 -04:00
dependabot[bot]
379282299a
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.4.0 to 1.6.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.4.0...sdk/azcore/v1.6.0 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-01 01:59:16 +00:00
dependabot[bot]
0c796dbd9b
build(deps): bump golang.org/x/oauth2 from 0.7.0 to 0.8.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.7.0 to 0.8.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.7.0...v0.8.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-01 01:58:29 +00:00
Daniel Gibson
f9dded83b3
doc/050_restore.rst: Mention that restic restore is fastest
...
Especially with lots of files (like when backing up git repos),
restic mount + cp -r can be really slow.
2023-05-31 22:44:36 +02:00
Michael Eischer
88a10a368f
Merge pull request #4342 from greatroar/errors
...
cmd: Don't check for errno == 0
2023-05-31 22:17:07 +02:00
Michael Eischer
9a7056a479
index: implement indexmap.grow() without random access
2023-05-30 20:13:33 +02:00
Michael Eischer
fc05e35a08
index: let indexmap.Each iterate in allocation order
...
Iterating through the indexmap according to the bucket order has the
problem that all indexEntries are accessed in random order which is
rather cache inefficient.
As we already keep a list of all allocated blocks, just iterate through
it. This allows iterating through a batch of indexEntries without random
memory accesses. In addition, the packID will likely remain similar
across multiple blobs as all blobs of a pack file are added as a single
batch.
2023-05-30 20:12:36 +02:00
Michael Eischer
f1c388c623
index: remove redundant storage of indexmap size
2023-05-30 20:11:53 +02:00
Michael Eischer
12141afbad
index: Allow inlining of HAT
2023-05-30 20:11:14 +02:00
Michael Eischer
fed33295c3
index: store indexEntries in hashed array tree
...
This data structure reduces the wasted memory to O(sqrt(n)). The
top-layer of the hashed array tree (HAT) also has a size of O(sqrt(n)),
which makes it cache efficient. The top-layer should be small enough to
easily fit into the CPU cache and thus only adds little overhead
compared to directly accessing an index entry via a pointer.
2023-05-29 00:24:15 +02:00
Michael Eischer
b217f38ee7
index: Remove pointers from within indexentrys
...
The indexEntry objects are now allocated in a separate array. References
to an indexEntry are now stored as array indices. This has the benefit
of allowing the garbage collector to ignore the indexEntry objects as
these do not contain pointers and are part of a single large allocation.
2023-05-29 00:24:15 +02:00
Michael Eischer
0c1240360d
index: add garbage collection benchmark
...
Allocates an index and repeatedly triggers the GC.
2023-05-29 00:23:04 +02:00
Michael Eischer
ffca602315
repository: Fix panic in benchmarkLoadIndex
2023-05-28 23:55:47 +02:00
greatroar
da419be43c
cache: Restructure New to remove redundant operations
...
New and its helpers used to create the cache directories several times
over. They now only do so once. The added test ensures that the cache is
produced in a consistent state when parts are deleted.
2023-05-27 10:32:08 +02:00
greatroar
8f1ca8fabe
cmd: Don't check for errno == 0
...
Maybe term.Restore's predecessor could return this, but the current
implementation never does.
2023-05-27 08:52:37 +02:00
Michael Eischer
f74dad2afb
Merge pull request #4340 from restic/least-privilege-ci
...
CI: least privilege for workflows
2023-05-26 20:29:05 +02:00
Michael Eischer
3e287afdbf
Merge pull request #4226 from Sohalt/gcs-region
...
gs: support other regions than us
2023-05-26 20:28:52 +02:00
Michael Eischer
06894484a1
shorten changelog title
2023-05-26 20:05:49 +02:00
Michael Eischer
6e5b42d5c4
CI: least privilege for workflows
2023-05-26 19:58:02 +02:00
sohalt
ed5b2c2c9b
gs: support other regions than us
2023-05-26 19:54:42 +02:00
Michael Eischer
13c32b0fbe
Merge pull request #4328 from MichaelEischer/cleanup-integration-tests
...
Cleanup integration tests
2023-05-26 19:49:16 +02:00
Michael Eischer
9747cef338
fix linter warnings
2023-05-26 19:38:44 +02:00
Michael Eischer
8e913e6d3a
repair index: always read Quiet flags from GlobalOptions passed as parameter
2023-05-26 19:38:44 +02:00
Michael Eischer
b93459cbb0
repair snapshots: use local copy of globalOptions to open repository
2023-05-26 19:38:44 +02:00
Michael Eischer
692f81ede8
cleanup prune integration test
2023-05-26 19:38:44 +02:00
Michael Eischer
7a268e4aba
always access stdout/stderr via globalOptions
2023-05-26 19:38:44 +02:00
Michael Eischer
4b3a0b4104
read JSON/Quiet flag from the passed in globalOptions
2023-05-26 19:38:44 +02:00
Michael Eischer
cebce52c16
test: add helper to capture stdout for integration tests
2023-05-26 19:38:44 +02:00
Michael Eischer
675a49a95b
Restructure integration tests
...
The tests are now split into individual files for each command. The
separation isn't perfect as many tests make use of multiple commands. In
particular `init`, `backup`, `check` and `list` are used by a larger
number of test cases.
Most tests now reside in files name cmd_<name>_integration_test.go. This
provides a certain indication which commands have significant test
coverage.
2023-05-26 19:38:43 +02:00
Michael Eischer
e2dba9f5c7
test: cleanup a some check calls
2023-05-26 19:34:57 +02:00
Michael Eischer
06fd6b54d7
test: print log output if testRunCheck fails
2023-05-26 19:34:57 +02:00
Michael Eischer
419e6f26b1
tests: Simplify checks that a specific number of snapshots exists
2023-05-26 19:34:57 +02:00
Michael Eischer
c3212ab6a6
test: use standard logging methods from testing for the test helpers
...
Use the logging methods from testing.TB to make use of tb.Helper(). This
allows the tests to log the filename and line number in which the test
helper was called. Previously the test helper was logged which is rarely
useful.
2023-05-26 19:34:57 +02:00
Michael Eischer
658aa4c0f7
Merge pull request #4339 from greatroar/ioctl
...
backend, termstatus: fix ioctl calls for 64-bit big-endian platforms
2023-05-26 19:34:14 +02:00
Michael Eischer
998cf5a7f8
Merge pull request #4333 from MichaelEischer/staticcheck
...
Update golangci-lint and resolve all errors
2023-05-26 19:34:00 +02:00
Michael Eischer
7eec91f841
add changelog
2023-05-26 19:23:51 +02:00
greatroar
51dc80be5b
backend: Use IoctlSetPointerInt for tcsetpgrp
...
This function casts its argument to int32 before passing it to the
system call, so that big-endian CPUs read the lower rather than the
upper 32 bits of the pid.
This also gets rid of the last import of "unsafe" in the Unix build.
I changed syscall to x/sys/unix while I was at it, to remove one more
import line. The constants and types there are aliases for their syscall
counterparts.
2023-05-25 17:31:51 +02:00
greatroar
ddbc0c1b37
termstatus: Fix IsProcessBackground for 64-bit big-endian Linux
...
Fixes #4223 . Verified with QEMU on linux/amd64.
2023-05-25 17:20:42 +02:00
Michael Eischer
ecbf8e055c
CI: upgrade golangci-lint and check full codebase
2023-05-18 21:17:53 +02:00
Michael Eischer
16ba237d8b
azure: Fix missing rate limiting
2023-05-18 21:17:53 +02:00
Michael Eischer
a466e945d9
stats: pass StatsOptions via parameter
2023-05-18 21:17:53 +02:00
Michael Eischer
03b9764bce
init/generate: check that no parameters are passed
2023-05-18 21:17:53 +02:00
Michael Eischer
22c9276719
rest: remove unused parameter
2023-05-18 21:17:53 +02:00
Michael Eischer
1e33b285c1
lock: remove unused parameter
2023-05-18 21:17:53 +02:00
Michael Eischer
c05f96e6b9
repair index: remove no longer used paramter
2023-05-18 21:17:53 +02:00
Michael Eischer
94752b7ee2
backup: Remove unused parameters from rejectFunc collection
2023-05-18 21:17:53 +02:00
Michael Eischer
0058745881
test: use parameter instead of hardcoded constant
2023-05-18 21:17:53 +02:00
Michael Eischer
a719d10e22
termstatus: remove unused parameter from helpers
2023-05-18 21:17:53 +02:00
Michael Eischer
b0a01ae68a
ui/backup: remove unused parameter from ProgressPrinter interface
2023-05-18 21:17:53 +02:00
Michael Eischer
472bf5184f
Replace lots of unused parameters with _
...
The parameters are required by the implemented function signature or interface.
2023-05-18 21:17:53 +02:00
Michael Eischer
d1a5ec7839
Rename unused testing parameter to _
...
The parameter is an additional marker that the test helper must only be
used for tests.
2023-05-18 21:17:53 +02:00
Michael Eischer
1514593f22
Remove unused context or testing parameters
2023-05-18 21:17:53 +02:00
Michael Eischer
5e4e268bdc
Use _ as parameter name for unused Context
...
The context is required by the implemented interface.
2023-05-18 21:15:45 +02:00
Michael Eischer
3252f60df5
Ignore deprecated imports for poly1305 and openpgp
2023-05-18 21:15:45 +02:00
Michael Eischer
2fa8b96843
Don't shadow builtins
2023-05-18 21:15:45 +02:00
Michael Eischer
7a01bd3b67
Remove unnecessary else block
2023-05-18 21:15:45 +02:00
Michael Eischer
319087c056
Remove redundant if ...; err != nil; return pattern
2023-05-18 21:15:45 +02:00
Michael Eischer
6ed73ed408
CI: remove deprecated linters
2023-05-18 21:15:45 +02:00
Michael Eischer
c832a492ac
fuse: use syscall errnos directly to fix deprecations
2023-05-18 21:15:45 +02:00
Michael Eischer
e01baeabba
Use either test or rtest to refer to internal test helpers
...
A single test file should not use both names.
2023-05-18 21:15:45 +02:00
Michael Eischer
bfc9c6c971
Merge pull request #4331 from MichaelEischer/fix-mount-failures
...
Hopefully fix `TestMount` failures
2023-05-18 20:22:57 +02:00
Michael Eischer
5773b86d02
repository: Push all usage of errors.Fatal out of the package
...
As the `Fatal` error type only includes a string, it becomes impossible
to inspect the contained error. This is for a example a problem for the
fuse implementation, which must be able to detect context.Canceled
errors.
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2023-05-18 17:27:41 +02:00
Michael Eischer
a013014c24
backup: return normal error if --group-by cannot be parsed
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2023-05-18 17:23:31 +02:00
Michael Eischer
f9850b79b5
rest/sftp: Remove redundant fatal from error message
...
This caused restic to exit with error messages like
`Fatal: parsing repository location failed: Fatal: sftp path [...]`
`Fatal: create repository at rest:http://localhost:12345/ failed: Fatal: config file already exists`
2023-05-18 17:22:26 +02:00
Michael Eischer
2f518b7241
Merge pull request #4304 from MichaelEischer/unlimited-lock-refresh
...
lock: Do not limit backend concurrency for lock files
2023-05-13 21:40:29 +02:00
Michael Eischer
49be202cb0
Merge pull request #4315 from MichaelEischer/restore-hardlink-progress
...
restore: Correctly account for hardlinks in progress bar
2023-05-13 21:37:40 +02:00
Michael Eischer
19ebc1b786
restore: Add basic test for progress bar accounting of hardlinks
2023-05-08 20:51:49 +02:00
Michael Eischer
23a122a901
restore: count files in the same way as the stats command
2023-05-08 20:51:49 +02:00
Michael Eischer
e77002f841
restore: correctly count hardlinks in progress bar
...
For hardlinked files, only the first instance of that file increases the
amount of bytes to restore. All later instances only increase the file
count but not the restore size.
2023-05-08 20:51:49 +02:00
Michael Eischer
d05f6211d1
lock: Do not limit backend concurrency for lock files
...
restic must be able to refresh lock files in time. However, large
uploads over slow connections can cause the lock refresh to be stuck
behind the large uploads and thus time out.
2023-05-08 19:04:46 +02:00
Michael Eischer
ee3c55ea3d
Merge pull request #2876 from aawsome/new-repair-command
...
Add repair command
2023-05-05 23:22:24 +02:00
Michael Eischer
db046c0acc
Merge pull request #4318 from MichaelEischer/status-output-truncation
...
Fix status output truncation
2023-05-05 23:09:59 +02:00
Michael Eischer
3e6a26e2e9
Merge pull request #4317 from MichaelEischer/debug-test-mount-failures
...
mount: enable debug logging for the flaky TestMount test
2023-05-05 23:08:03 +02:00
greatroar
65c5e511a1
ui/termstatus: Add test for Terminal.SetStatus
2023-05-05 22:51:41 +02:00
Michael Eischer
6d10c655a0
termstatus: test status line sanitization
2023-05-05 22:51:41 +02:00
Michael Eischer
bb40e49e75
ui/termstatus: Fix truncation of status output
...
The last line was not truncated as expected
2023-05-05 22:45:52 +02:00
Michael Eischer
fefe15d7a1
Merge pull request #4316 from MichaelEischer/fix-check-modified-data
...
Fix flaky TestCheckerModifiedData test
2023-05-05 22:38:28 +02:00
Michael Eischer
78e5aa6d30
repair snapshots: add basic tests
2023-05-04 23:00:46 +02:00
Michael Eischer
1cb11ad8ad
mount: enable debug logging for the flaky TestMount test
...
The test case fails from time to time with an Input/Output error while
trying to access the snapshots directory.
2023-05-01 18:03:17 +02:00
Michael Eischer
90a663c94f
Merge pull request #4308 from greatroar/tar-header
...
dump: Report filename with tar.ErrFieldTooLong
2023-05-01 17:30:31 +02:00
Michael Eischer
88a7231217
report snapshot id if loading failed
2023-05-01 17:24:13 +02:00
Michael Eischer
c0627dc80d
check: Fix flaky TestCheckerModifiedData
...
The test had a 4% chance of not modified the data read from the
repository, in which case the test would fail. Change the data
manipulation to just modified each read operation.
2023-05-01 17:18:19 +02:00
Michael Eischer
e71367e6b9
repair snapshots: update changelog
2023-05-01 16:06:17 +02:00
Michael Eischer
5aa37acdaa
repair snapshots: cleanup command help
2023-05-01 16:06:17 +02:00
Michael Eischer
9c64a95df8
doc: rewrite troubleshooting section
2023-05-01 16:06:17 +02:00
Michael Eischer
7c8dd61e8c
repair snapshots: cleanup warnings
2023-05-01 15:22:30 +02:00
Michael Eischer
f6cc10578d
repair snapshots: Always sanitize file nodes
...
If the node for a file is intact, this is a no-op.
2023-05-01 15:22:18 +02:00
Michael Eischer
4ce87a7f64
repair snapshots: port to filterAndReplaceSnapshot
...
The previous approach of rewriting all snapshots first, then flushing
the repository data and finally removing old snapshots has the downside
that an interrupted command execution leaves behind broken snapshots as
not all new data is already flushed.
2023-05-01 15:22:03 +02:00
Michael Eischer
e17ee40a31
repair snapshots: Port to use walker.TreeRewriter
2023-05-01 15:20:36 +02:00
Michael Eischer
1bd1f3008d
walker: extend TreeRewriter to support snapshot repairing
...
This adds support for caching already rewritten trees, handling of load
errors and disabling the check that the serialization doesn't lead to
data loss.
2023-05-01 15:20:24 +02:00
Michael Eischer
38dac78180
walker: restructure FilterTree into TreeRewriter
...
The more generic RewriteNode callback replaces the SelectByName and
PrintExclude functions. The main part of this change is a preparation to
allow using the TreeRewriter for the `repair snapshots` command.
2023-05-01 15:20:12 +02:00
Michael Eischer
bc2399fbd9
walker: recurse into directory based on node type
...
A broken directory might also not have a subtree.
2023-05-01 15:20:00 +02:00
Michael Eischer
1a9705fc95
walker: Simplify change detection in FilterTree
...
Now the rewritten tree is always serialized which makes sure that we
don't accidentally miss any relevant changes.
2023-05-01 15:19:48 +02:00
Michael Eischer
8c4caf09a8
repair snapshots: Do not rename repaired files
...
The files in a tree must be sorted in lexical order. However, this
cannot be guaranteed when appending a filename suffix. For two files
file, file.rep
where "file" is broken, this would result in
file.repaired, file.rep
which is no longer sorted.
In addition, adding a filename suffix is also prone to filename
collisions which would require a rather complex search for a
collision-free name in order to work reliably.
2023-05-01 15:19:36 +02:00
Michael Eischer
375189488c
rewrite: prepare for code sharing with rewrite snapshots
2023-05-01 15:19:24 +02:00
Michael Eischer
903651c719
repair snapshots: partially synchronize code with rewrite command
...
Simplify CLI options:
* Rename "DeleteSnapshots" to "Forget"
* Replace "AddTag" and "Append" with hardcoded values
Change output and snapshot modifications to be more in line with the
"rewrite" command.
2023-05-01 15:19:11 +02:00
Michael Eischer
118d599d0a
Rename 'rebuild-index' to 'repair index'
...
The old name still works, but is deprecated.
2023-05-01 15:16:44 +02:00
Michael Eischer
db459eda21
move to subcommand
2023-05-01 15:15:48 +02:00
Michael Eischer
a14a63cd29
modernize code
2023-05-01 15:12:09 +02:00
Alexander Weiss
947f0c345e
correct typos
2023-05-01 14:56:42 +02:00
Alexander Weiss
d23a2e1925
better error handling and correct nil tree behavior
2023-05-01 14:56:42 +02:00
Alexander Weiss
08ae708b3b
make linter happy
2023-05-01 14:56:42 +02:00
Alexander Weiss
99a05d5ab2
Update troubleshooting documentation
2023-05-01 14:56:42 +02:00
Alexander Weiss
6557f36f61
Add changelog and docu for #2876
2023-05-01 14:56:42 +02:00
Alexander Weiss
5f58797ba7
Add repair command
2023-05-01 14:56:42 +02:00
Alexander Weiss
9cef6b4c69
Add troubleshooting documentation
2023-05-01 14:56:42 +02:00
Michael Eischer
8a78a042db
Merge pull request #4310 from restic/dependabot/go_modules/github.com/klauspost/compress-1.16.5
...
build(deps): bump github.com/klauspost/compress from 1.16.0 to 1.16.5
2023-05-01 14:35:57 +02:00
Michael Eischer
b491af2b57
Merge pull request #4311 from restic/dependabot/go_modules/golang.org/x/crypto-0.8.0
...
build(deps): bump golang.org/x/crypto from 0.7.0 to 0.8.0
2023-05-01 14:35:37 +02:00
dependabot[bot]
d747a9c401
build(deps): bump golang.org/x/crypto from 0.7.0 to 0.8.0
...
Bumps [golang.org/x/crypto](https://github.com/golang/crypto ) from 0.7.0 to 0.8.0.
- [Release notes](https://github.com/golang/crypto/releases )
- [Commits](https://github.com/golang/crypto/compare/v0.7.0...v0.8.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/crypto
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-01 11:15:38 +00:00
dependabot[bot]
888c1ae63a
build(deps): bump github.com/klauspost/compress from 1.16.0 to 1.16.5
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.16.0 to 1.16.5.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.16.0...v1.16.5 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-01 11:07:29 +00:00
Michael Eischer
5eeb257c95
Merge pull request #4312 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.52
...
build(deps): bump github.com/minio/minio-go/v7 from 7.0.50 to 7.0.52
2023-05-01 13:00:48 +02:00
Michael Eischer
ae6729cf89
Merge pull request #4309 from restic/dependabot/go_modules/golang.org/x/oauth2-0.7.0
...
build(deps): bump golang.org/x/oauth2 from 0.6.0 to 0.7.0
2023-05-01 12:58:55 +02:00
dependabot[bot]
6abd494915
build(deps): bump github.com/minio/minio-go/v7 from 7.0.50 to 7.0.52
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.50 to 7.0.52.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.50...v7.0.52 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-01 01:59:50 +00:00
dependabot[bot]
7147a54ceb
build(deps): bump golang.org/x/oauth2 from 0.6.0 to 0.7.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/oauth2/releases )
- [Commits](https://github.com/golang/oauth2/compare/v0.6.0...v0.7.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-01 01:59:10 +00:00
Michael Eischer
8d971172c4
Merge pull request #4306 from MichaelEischer/document-restic-compression-env
...
Document that the compression mode can be set via $RESTIC_COMPRESSION
2023-04-30 18:26:10 +02:00
Michael Eischer
37d0e323eb
doc: update help output in manual_rest.rst
2023-04-30 16:12:29 +02:00
Michael Eischer
face5bd7f7
Document that the compression mode can be set via $RESTIC_COMPRESSION
2023-04-30 16:11:53 +02:00
Michael Eischer
1daf928a77
Merge pull request #4305 from MichaelEischer/stracktrace-for-windows
...
Print stacktrace in SIGINT handler if RESTIC_DEBUG_STACKTRACE_SIGINT set
2023-04-30 16:08:58 +02:00
Michael Eischer
37bab08181
Merge pull request #4302 from MichaelEischer/logging-cleanup
...
Tweak debug logs
2023-04-30 16:08:08 +02:00
Michael Eischer
6dc2324d2e
Merge pull request #4301 from MichaelEischer/less-restore-memory
...
restore: slightly reduce memory usage while restoring files
2023-04-30 16:07:43 +02:00
Michael Eischer
850cd9aace
Merge pull request #4299 from MichaelEischer/update-scripting-docs
...
doc: recommend `cat config` to check if a repository exists
2023-04-30 16:04:59 +02:00
greatroar
b50ff04cf3
dump: Report filename with tar.ErrFieldTooLong
...
Updates #4307 .
2023-04-27 11:45:41 +02:00
Michael Eischer
c8641f4479
Merge branch 'patch-release'
2023-04-24 20:44:38 +02:00
Alexander Neumann
ac7ac0cb97
Set development version for 0.15.2
2023-04-24 20:28:37 +02:00
Alexander Neumann
be8be3397c
Add version for 0.15.2
2023-04-24 20:28:37 +02:00
Alexander Neumann
db6b4f8912
Update manpages and auto-completion
2023-04-24 20:28:37 +02:00
Alexander Neumann
1f3f042f32
Generate CHANGELOG.md for 0.15.2
2023-04-24 20:28:24 +02:00
Alexander Neumann
0aaa4e6cbe
Prepare changelog for 0.15.2
2023-04-24 20:28:24 +02:00
Leo R. Lundgren
0bac935dac
doc: Polish changelogs
2023-04-23 22:10:04 +02:00
Michael Eischer
306a29980a
Print stacktrace in SIGINT handler if RESTIC_DEBUG_STACKTRACE_SIGINT set
...
The builtin mechanism to capture a stacktrace in Go is to send a SIGQUIT
to the running process. However, this mechanism is not avaiable on
Windows. Thus, tweak the SIGINT handler to dump a stacktrace if the
environment variable `RESTIC_DEBUG_STACKTRACE_SIGINT` is set.
2023-04-23 15:50:40 +02:00
Michael Eischer
1e6e9f9bd0
tweak changelogs
2023-04-23 12:46:55 +02:00
greatroar
f342db7666
ui/termstatus: Quote funny filenames
...
Fixes #2260 , #4191 .
2023-04-23 12:45:40 +02:00
Michael Eischer
41cc320145
Merge pull request #4166 from MichaelEischer/cache-mkdirall
...
cache: Create missing intermediate directories when caching a file
2023-04-23 11:44:18 +02:00
Michael Eischer
cdb0fb9c06
tweak debug logs
2023-04-23 11:38:06 +02:00
Michael Eischer
94cbc6392d
restore: slightly reduce memory usage while restoring files
...
The information which target files are contained in a pack file is no
longer necessary after processing a pack.
2023-04-23 11:33:21 +02:00
Michael Eischer
78a1757e5a
Cancel current command if cache becomes unusable
...
If the cache suddenly disappears, the current command will now fail.
2023-04-23 11:31:15 +02:00
Michael Eischer
22562d2132
Merge pull request #4300 from MichaelEischer/less-flaky-tests
...
Increase timeouts for lock refresh tests
2023-04-23 11:24:59 +02:00
Michael Eischer
51d823348d
Merge pull request #4286 from MichaelEischer/backend-cleanup-logging
...
Normalize backend logging and connection limiting
2023-04-23 11:24:26 +02:00
Michael Eischer
831f593b87
backend/sema: Add tests
2023-04-22 13:20:20 +02:00
Michael Eischer
179e11c2ae
Increase timeouts for lock refresh tests
...
When saving files to the local backend, in some cases the used fsync
calls are slow enough to cause the tests to time out. Thus, increase the
test timeouts as a stopgap measure until we can use the mem backend for
these tests.
2023-04-22 12:45:59 +02:00
Michael Eischer
ebba233a3a
backend/sema: rename constructor to NewBackend
2023-04-22 12:32:57 +02:00
rawtaz
8479390d7c
Merge pull request #3939 from Fabien-jrt/resticprofile
...
doc: Add resticprofile to scheduling section
2023-04-22 01:24:34 +02:00
Fabien-Jrt
756f43d5f9
doc: Add resticprofile to scheduling section
2023-04-22 01:04:48 +02:00
Michael Eischer
affd04c125
doc: recommend `cat config` to check if a repository exists
...
This will be much faster if a large number of snapshots exists.
2023-04-21 22:56:31 +02:00
Michael Eischer
f27750e270
backend/sema: rename type to connectionLimitedBackend
2023-04-21 22:53:09 +02:00
Michael Eischer
1dd873b706
Merge pull request #4293 from MichaelEischer/fix-groups-by
...
Fix parent snapshot selection for relative paths
2023-04-21 22:47:03 +02:00
Michael Eischer
7a60d9e54f
Merge pull request #4288 from MichaelEischer/log-warnings-to-debug-log
...
Add warnings via Warnf to the debug log
2023-04-21 22:39:00 +02:00
Michael Eischer
3001dd8c2b
Add test to verify that the backup parent is correctly selected
2023-04-21 22:35:02 +02:00
Michael Eischer
4503aea0df
Merge pull request #4296 from MichaelEischer/rewrite-reorder-output
...
rewrite: log snapshot saved before removal of the old snapshot
2023-04-19 22:00:32 +02:00
Michael Eischer
09cddb8927
rewrite: log snapshot saved before removal of the old snapshot
...
The snapshot was already saved before removing the old snapshot. Only
the log messages were printed in the wrong order.
2023-04-17 21:00:45 +02:00
Michael Eischer
913eab3361
Merge pull request #4234 from thndrbrrr/forget-opts-neg1-means-forever-issue-2565
...
restic forget --keep-* options will interpret -1 as "forever"
2023-04-14 23:18:47 +02:00
Michael Eischer
a9c7c12276
Merge pull request #4265 from HeikoSchlittermann/chg/progress-to-stderr
...
generate: do not write progress to STDOUT
2023-04-14 23:18:23 +02:00
Michael Eischer
85eef232e6
Merge pull request #4232 from thndrbrrr/copy-chunker-params-verbose-msg
...
init: Add --copy-chunker-params verbose msg
2023-04-14 23:17:56 +02:00
Michael Eischer
07a44a88f2
Fix snapshot filtering for relative paths in the backup command
...
The snapshot filtering internally converts relative paths to absolute
ones to ensure that the parent snapshots selection works for backups of
relative paths.
2023-04-14 22:58:34 +02:00
Michael Eischer
48e065d971
Sync dependency upgrades from master
...
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob still uses v0.5.1
as upgrading it would increase the minimum Go version on Solaris to
1.20.
2023-04-14 22:58:34 +02:00
Michael Eischer
a06d927dce
Merge pull request #4291 from greatroar/widechars
...
ui/termstatus: Optimize Truncate
2023-04-14 22:48:34 +02:00
Michael Eischer
fd3ed9e2f4
Merge pull request #4192 from greatroar/quote
...
ui/termstatus: Quote funny filenames
2023-04-14 22:39:09 +02:00
Michael Eischer
6042df075f
migrations: Fix S3 backend detection
2023-04-14 22:32:16 +02:00
Michael Eischer
c934c99d41
gs: replace usage of context.Background()
2023-04-14 22:32:15 +02:00
Michael Eischer
616926d2c1
gs: use IsNotExist to check error
2023-04-14 22:32:15 +02:00
Michael Eischer
05abc6d6f5
backend: deduplicate implementation of Delete() method
2023-04-14 22:32:15 +02:00
Michael Eischer
45244fdf68
backend: remove parameter validation tests
...
These parameter validations have been factored out into
SemaphoreBackend.
2023-04-14 22:32:15 +02:00
Michael Eischer
803640ba4b
backend: remove a few unnecessary debug logs
2023-04-14 22:32:15 +02:00
Michael Eischer
8e1e3844aa
backend: factor out connection limiting and parameter validation
...
The SemaphoreBackend now uniformly enforces the limit of concurrent
backend operations. In addition, it unifies the parameter validation.
The List() methods no longer uses a semaphore. Restic already never runs
multiple list operations in parallel.
By managing the semaphore in a wrapper backend, the sections that hold a
semaphore token grow slightly. However, the main bottleneck is IO, so
this shouldn't make much of a difference.
The key insight that enables the SemaphoreBackend is that all of the
complex semaphore handling in `openReader()` still happens within the
original call to `Load()`. Thus, getting and releasing the semaphore
tokens can be refactored to happen directly in `Load()`. This eliminates
the need for wrapping the reader in `openReader()` to release the token.
2023-04-14 22:32:15 +02:00
Michael Eischer
8b5ab5b59f
dryrun: fix outdated comments
2023-04-14 22:32:15 +02:00
Michael Eischer
4703473ec5
backend: extract most debug logs into logger backend
2023-04-14 22:32:15 +02:00
Michael Eischer
8bfc2519d7
backend: Deduplicate sanity checks for parameters of Load() method
...
The check is now handled by backend.DefaultLoad. This also guarantees
consistent behavior across all backends.
2023-04-14 22:32:15 +02:00
Michael Eischer
ba16904eed
backup: Add test to verify parent snapshot selection for relative paths
2023-04-14 22:21:43 +02:00
Michael Eischer
2841a87cc6
Fix snapshot filtering for relative paths in the backup command
...
The snapshot filtering internally converts relative paths to absolute
ones to ensure that the parent snapshots selection works for backups of
relative paths.
2023-04-14 21:53:55 +02:00
Michael Eischer
fab4a8a4d2
Properly initialize the --group-by option for backup tests
2023-04-14 21:53:01 +02:00
greatroar
3b24c15c3d
fuse: Mix inode hashes in a non-symmetric way
...
Since 0.15 (#4020 ), inodes are generated as hashes of names, xor'd with
the parent inode. That means that the inode of a/b/b is
h(a/b/b) = h(a) ^ h(b) ^ h(b) = h(a).
I.e., the grandchild has the same inode as the grandparent. GNU find
trips over this because it thinks it has encountered a loop in the
filesystem, and fails to search a/b/b. This happens more generally when
the same name occurs an even number of times.
Fix this by multiplying the parent by a large prime, so the combining
operation is not longer symmetric in its arguments. This is what the FNV
hash does, which we used prior to 0.15. The hash is now
h(a/b/b) = h(b) ^ p*(h(b) ^ p*h(a))
Note that we already ensure that h(x) is never zero.
Collisions can still occur, but they should be much less likely to occur
within a single path.
Fixes #4253 .
2023-04-14 20:50:39 +02:00
greatroar
4304e01ca2
fuse: Report fuse.Attr.Blocks correctly
...
Fixes #4239 .
2023-04-14 20:50:08 +02:00
Torben Giesselmann
6d6c04abef
forget: Simplify usage text
2023-04-14 10:05:23 -07:00
greatroar
49e32f3f8a
ui/termstatus: Optimize Truncate
...
x/text/width.LookupRune has to re-encode its argument as UTF-8,
while LookupString operates on the UTF-8 directly.
The uint casts get rid of a bounds check.
Benchmark results, with b.ResetTimer introduced first:
name old time/op new time/op delta
TruncateASCII-8 69.7ns ± 1% 55.2ns ± 1% -20.90% (p=0.000 n=20+18)
TruncateUnicode-8 350ns ± 1% 171ns ± 1% -51.05% (p=0.000 n=20+19)
2023-04-14 11:13:39 +02:00
greatroar
9412f37e50
ui/termstatus: Quote funny filenames
...
Fixes #2260 , #4191 .
2023-04-14 09:36:02 +02:00
Ian Muge
593eb710b4
added changelog
2023-04-13 22:54:36 +02:00
greatroar
97274ecabd
cmd, restic: Refactor and fix snapshot filtering
...
This turns snapshotFilterOptions from cmd into a restic.SnapshotFilter
type and makes restic.FindFilteredSnapshot and FindFilteredSnapshots
methods on that type. This fixes #4211 by ensuring that hosts and paths
are named struct fields instead of unnamed function arguments in long
lists of such.
Timestamp limits are also included in the new type. To avoid too much
pointer handling, the convention is that time zero means no limit.
That's January 1st, year 1, 00:00 UTC, which is so unlikely a date that
we can sacrifice it for simpler code.
2023-04-13 22:51:45 +02:00
Michael Eischer
74f7dd0b38
Make help for --verbose less confusing
...
The output is now
```
-v, --verbose be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)
```
instead of
```
-v, --verbose n be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)
```
2023-04-13 22:46:37 +02:00
Michael Eischer
21ad357c10
add linux/riscv64 builds
2023-04-13 22:45:35 +02:00
Heiko Schlittermann (HS12-RIPE)
087cf7e114
generate: write progress to STDOUT if this is a terminal
...
This allows usage as:
eval "$(restic generated --bash-completion /dev/stdout)"
2023-04-12 22:39:59 +02:00
Michael Eischer
e604939e72
Debug log status code if execution is interrupted
...
Currently, there is no clear indication in the debug log whether restic
exited or not.
2023-04-12 21:58:35 +02:00
Michael Eischer
37aca6bec0
Add warnings via Warnf to the debug log
2023-04-12 21:37:37 +02:00
Michael Eischer
bdcafbc11c
Merge pull request #4282 from TorstenC/patch-1
...
040_backup.rst line 228 - #4280 --exclude-caches
2023-04-11 22:34:16 +02:00
Torsten C
ba33e41068
040_backup.rst line 228 - #4280 --exclude-caches
...
Proposal for issue #4280
2023-04-11 22:33:10 +02:00
Michael Eischer
4661f45a8c
Merge pull request #4220 from Skaronator/master
...
Add jq to container image
2023-04-11 22:26:02 +02:00
Michael Eischer
2091fc0dde
Merge pull request #4107 from jooola/feature-wait-for-unlock
...
Add a global option --retry-lock
2023-04-11 22:21:01 +02:00
Michael Eischer
9238dcc81a
Merge pull request #4201 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob-1.0.0
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 0.5.1 to 1.0.0
2023-04-11 22:20:38 +02:00
Torben Giesselmann
71537da4b0
init: Add --copy-chunker-params verbose msg
2023-04-09 14:59:07 -07:00
Torben Giesselmann
ce51d2f3c0
forget: Update usage text
2023-04-09 12:59:15 -07:00
Torben Giesselmann
403b7ca2be
forget: Update documentation
2023-04-09 12:57:37 -07:00
Torben Giesselmann
309cf0586a
Add changelog entry
2023-04-09 12:05:15 -07:00
Torben Giesselmann
9f9e91eb0d
Fix comment to comply with linter
2023-04-09 11:47:10 -07:00
Michael Eischer
1f43003cc1
Merge branch 'master' into feature-wait-for-unlock
2023-04-07 20:32:21 +02:00
Michael Eischer
8ce5f29758
lock: increase test timeout tolerances to avoid test failures
2023-04-07 20:07:05 +02:00
Michael Eischer
8afc117aa3
Merge pull request #4285 from MichaelEischer/debug-lock-refresh-failures
...
lock: Improve debug logging in the test
2023-04-07 20:06:15 +02:00
Michael Eischer
cf1cc1fb72
lock: Print stacktrace if TestLockSuccessfulRefresh fails
2023-04-07 19:51:31 +02:00
Michael Eischer
64233ca0a7
lock: Improve debug logging in the test
2023-04-07 19:43:00 +02:00
jo
ea59896bd6
Add a global option --retry-lock
...
Fixes restic#719
If the option is passed, restic will wait the specified duration of time
and retry locking the repo every 10 seconds (or more often if the total
timeout is relatively small).
- Play nice with json output
- Reduce wait time in lock tests
- Rework timeout last attempt
- Reduce test wait time to 0.1s
- Use exponential back off for the retry lock
- Don't pass gopts to lockRepo functions
- Use global variable for retry sleep setup
- Exit retry lock on cancel
- Better wording for flag help
- Reorder debug statement
- Refactor tests
- Lower max sleep time to 1m
- Test that we cancel/timeout in time
- Use non blocking sleep function
- Refactor into minDuration func
Co-authored-by: Julian Brost <julian@0x4a42.net>
2023-04-07 16:24:14 +02:00
Michael Eischer
806a0cdce3
bump minimum go version on Solaris to 1.20
2023-04-07 15:40:57 +02:00
Michael Eischer
faa83db9e4
azure: Adapt code to API change
2023-04-07 15:05:55 +02:00
dependabot[bot]
9358a5fb37
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go ) from 0.5.1 to 1.0.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/armcore/v0.5.1...v1.0 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-04-07 11:03:25 +00:00
Michael Eischer
71c9516b26
Merge pull request #4247 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.4.0
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.3.1 to 1.4.0
2023-04-07 13:01:04 +02:00
Michael Eischer
17ac91fba3
Merge pull request #4283 from restic/dependabot/go_modules/google.golang.org/api-0.116.0
...
build(deps): bump google.golang.org/api from 0.114.0 to 0.116.0
2023-04-07 12:58:54 +02:00
Michael Eischer
26a3c47c5c
Merge pull request #4255 from greatroar/fuse-hash
...
fuse: Mix inode hashes in a non-symmetric way
2023-04-07 12:56:57 +02:00
Michael Eischer
7896e50301
Merge pull request #4271 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.50
...
build(deps): bump github.com/minio/minio-go/v7 from 7.0.49 to 7.0.50
2023-04-07 12:47:30 +02:00
dependabot[bot]
9584cbda90
build(deps): bump google.golang.org/api from 0.114.0 to 0.116.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.114.0 to 0.116.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.114.0...v0.116.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-04-07 10:27:38 +00:00
dependabot[bot]
cacc48fc09
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.3.1 to 1.4.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.3.1...sdk/azcore/v1.4.0 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-04-07 10:27:30 +00:00
Michael Eischer
7fbaca577b
Merge pull request #4272 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.30.1
...
build(deps): bump cloud.google.com/go/storage from 1.29.0 to 1.30.1
2023-04-07 12:26:20 +02:00
Michael Eischer
1f9dd84d1e
Merge pull request #3991 from mark-herrmann/feature/restore-progress
...
restore: show progress
2023-04-07 12:24:55 +02:00
Michael Eischer
c8ff5592b8
Merge pull request #4273 from ducalex/gitignore-restic-exe
...
Added restic.exe to the .gitignore
2023-04-07 12:10:09 +02:00
Mark Herrmann
f875a8843d
restore: Add progress bar
...
Co-authored-by: Mark Herrmann <mark.herrmann@mailbox.org>
2023-04-07 12:08:23 +02:00
Alex Duchesne
0ed5c20c57
Added restic.exe to the .gitignore
2023-03-28 23:07:25 -04:00
dependabot[bot]
21edbdc3ac
build(deps): bump cloud.google.com/go/storage from 1.29.0 to 1.30.1
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.29.0 to 1.30.1.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.29.0...spanner/v1.30.1 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-26 20:14:09 +00:00
dependabot[bot]
220d937975
build(deps): bump github.com/minio/minio-go/v7 from 7.0.49 to 7.0.50
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.49 to 7.0.50.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.49...v7.0.50 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-26 20:13:34 +00:00
rawtaz
5f13bbc118
Merge pull request #4270 from MichaelEischer/monthly-dependabot
...
Switch back to monthly dependabot updates
2023-03-26 22:09:21 +02:00
Michael Eischer
9672670756
Merge pull request #4240 from greatroar/fuse-blocks
...
fuse: Report fuse.Attr.Blocks correctly
2023-03-26 21:57:53 +02:00
Michael Eischer
5ac24a9744
Switch back to monthly dependabot updates
2023-03-26 21:55:27 +02:00
Michael Eischer
99e247caa6
Merge pull request #4259 from restic/dependabot/github_actions/actions/setup-go-4
...
build(deps): bump actions/setup-go from 3 to 4
2023-03-26 21:53:47 +02:00
Michael Eischer
0c705e07db
Merge pull request #4249 from restic/dependabot/go_modules/golang.org/x/oauth2-0.6.0
...
build(deps): bump golang.org/x/oauth2 from 0.5.0 to 0.6.0
2023-03-26 10:38:53 +02:00
Michael Eischer
024d01d85b
Merge pull request #4266 from restic/linkfix
...
doc: Correct broken links
2023-03-22 22:59:54 +01:00
Leo R. Lundgren
0666fa11b8
doc: Correct broken links
2023-03-22 17:37:58 +01:00
rawtaz
caa0e89114
Merge pull request #4246 from greatroar/doc
...
doc: Fix links, focus less on implementation details
2023-03-22 14:47:31 +01:00
rawtaz
46a7072f3f
Merge pull request #4264 from GuitarBilly/patch-1
...
Update 110_talks.rst - Add changelog podcast.
2023-03-22 14:13:40 +01:00
GuitarBilly
1f12915b0c
Update 110_talks.rst
...
Update 110_talks.rst
align date to 2nd of April as agreed.
( match podcast publication date )
2023-03-22 13:54:27 +01:00
greatroar
a0885d5d69
fuse: Mix inode hashes in a non-symmetric way
...
Since 0.15 (#4020 ), inodes are generated as hashes of names, xor'd with
the parent inode. That means that the inode of a/b/b is
h(a/b/b) = h(a) ^ h(b) ^ h(b) = h(a).
I.e., the grandchild has the same inode as the grandparent. GNU find
trips over this because it thinks it has encountered a loop in the
filesystem, and fails to search a/b/b. This happens more generally when
the same name occurs an even number of times.
Fix this by multiplying the parent by a large prime, so the combining
operation is not longer symmetric in its arguments. This is what the FNV
hash does, which we used prior to 0.15. The hash is now
h(a/b/b) = h(b) ^ p*(h(b) ^ p*h(a))
Note that we already ensure that h(x) is never zero.
Collisions can still occur, but they should be much less likely to occur
within a single path.
Fixes #4253 .
2023-03-21 17:33:18 +01:00
dependabot[bot]
5dccab701a
build(deps): bump actions/setup-go from 3 to 4
...
Bumps [actions/setup-go](https://github.com/actions/setup-go ) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases )
- [Commits](https://github.com/actions/setup-go/compare/v3...v4 )
---
updated-dependencies:
- dependency-name: actions/setup-go
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 02:18:58 +00:00
Torben Giesselmann
5069c9edd9
Represent -1 as "all" in ExpirePolicy's Stringer
2023-03-15 15:07:51 -07:00
Torben Giesselmann
1a584cb16e
Refactor policy sum calculation & duration parsing
...
- Convert policy sum calculation to function and move to tests.
- Remove parseDuration(...) and use ParseDurationOrPanic(...) instead.
2023-03-14 19:29:08 -07:00
Torben Giesselmann
84ede6ad7a
forget: Prevent neg. values in --keep-within* opts
2023-03-14 19:20:03 -07:00
Torben Giesselmann
b7f03d01b8
Add helper function for Duration parsing
...
Tests in cmd_forget_test.go need the same convenience function
that was implemented in snapshot_policy_test.go.
Function parseDuration(...) was moved to testing.go and renamed to
ParseDurationOrPanic(...).
2023-03-14 19:16:24 -07:00
dependabot[bot]
eaceaca113
build(deps): bump golang.org/x/oauth2 from 0.5.0 to 0.6.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/golang/oauth2/releases )
- [Commits](https://github.com/golang/oauth2/compare/v0.5.0...v0.6.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-13 02:07:39 +00:00
greatroar
a7ac9a4769
doc: Fix some links, focus less on implementation details
...
The godoc for filepath.Match has the syntax, which is what is important
for writing patterns. Use pkg.go.dev instead of golang.org/pkg.
For #4245 . Not all links fixed yet.
2023-03-13 00:23:15 +01:00
Michael Eischer
23ed03a267
Merge pull request #4243 from restic/update-go
...
Run tests with Go 1.20
2023-03-12 18:15:54 +01:00
Alexander Neumann
aac8c5a7ca
Merge pull request #4244 from MichaelEischer/fix-doc-typos
...
Fix documentation typos and build with Sphinx 6
2023-03-12 15:04:26 +01:00
Michael Eischer
7c8a401d97
Merge pull request #4176 from LXGaming/fix/impostor
...
Fix scan_finished JSON MessageType
2023-03-12 12:14:40 +01:00
Alexander Neumann
d83332315c
Update golangci-lint
2023-03-12 12:05:59 +01:00
Alexander Neumann
c2703e5024
Increase timeout for golangci-lint
2023-03-12 12:02:22 +01:00
Michael Eischer
eb7dbc88b5
Merge pull request #4210 from aawsome/patch-1
...
docs: Corrections/extra information in design.rst
2023-03-12 12:00:25 +01:00
Michael Eischer
e02aadf1d2
doc: use extlink for links to issues
2023-03-12 11:55:36 +01:00
Michael Eischer
c4e6b198ae
doc: fix building with Sphinx 6
2023-03-12 11:45:26 +01:00
Michael Eischer
f47c8eebb7
doc: fix broken link in rewrite command docs
2023-03-12 11:44:56 +01:00
Alexander Neumann
996e2ac7c5
Run tests with Go 1.20
2023-03-12 11:41:22 +01:00
Michael Eischer
a67d3781a3
doc: apply review comments and improve link formatting
2023-03-12 11:37:37 +01:00
Michael Eischer
153a73ebba
Merge pull request #4242 from MichaelEischer/fix-mixed-index-in-design-docs
...
doc: Remove mixed pack files from index example
2023-03-12 11:24:21 +01:00
Michael Eischer
f9d6e3a035
Merge pull request #4236 from restic/dependabot/go_modules/golang.org/x/crypto-0.7.0
...
build(deps): bump golang.org/x/crypto from 0.5.0 to 0.7.0
2023-03-11 20:42:38 +01:00
dependabot[bot]
51656e8764
build(deps): bump golang.org/x/crypto from 0.5.0 to 0.7.0
...
Bumps [golang.org/x/crypto](https://github.com/golang/crypto ) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/crypto/releases )
- [Commits](https://github.com/golang/crypto/compare/v0.5.0...v0.7.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/crypto
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-11 19:17:59 +00:00
Michael Eischer
9e23200bff
Merge pull request #4219 from ianmuge/feat_upgrade_minio_for_new_region
...
(feat) Upgrade Minio to 7.0.49
2023-03-11 20:17:03 +01:00
Michael Eischer
b884643b40
Merge pull request #4224 from restic/dependabot/go_modules/github.com/klauspost/compress-1.16.0
...
build(deps): bump github.com/klauspost/compress from 1.15.15 to 1.16.0
2023-03-11 20:14:55 +01:00
Ian Muge
5be4845710
added changelog
2023-03-11 20:04:12 +01:00
Ian Muge
cfa3c6abc5
upgrade Minio to 7.0.49
2023-03-11 20:04:12 +01:00
Michael Eischer
f499e66032
Merge pull request #4235 from restic/dependabot/go_modules/google.golang.org/api-0.111.0
...
build(deps): bump google.golang.org/api from 0.109.0 to 0.111.0
2023-03-11 20:02:03 +01:00
Michael Eischer
00575ecffe
doc: Remove mixed pack files from index example
...
The example given for the format of an index shows a mixed pack file.
Mixing tree and data blobs has been deprecated for a long time. Thus,
change the pack to only contain "data" blobs.
2023-03-11 19:10:31 +01:00
greatroar
cb5694d136
fuse: Report fuse.Attr.Blocks correctly
...
Fixes #4239 .
2023-03-07 22:14:07 +01:00
dependabot[bot]
100b06d806
build(deps): bump google.golang.org/api from 0.109.0 to 0.111.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.109.0 to 0.111.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.109.0...v0.111.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-06 02:11:38 +00:00
Torben Giesselmann
667536cea4
forget: Allow neg. values in "--keep-within*" opts
2023-03-05 14:18:08 -08:00
Torben Giesselmann
ba183c44c3
forget: Add test with regular and "forever" opts
2023-03-05 13:56:16 -08:00
Torben Giesselmann
32e6a438be
forget: Add test for "keep all hourly snapshots"
2023-03-04 22:13:34 -08:00
Torben Giesselmann
b77b0749fa
forget: Treat -1 as forever for all "last n" opts
2023-03-04 19:01:37 -08:00
Torben Giesselmann
6aca7dac21
forget: Verify forget opts
2023-03-03 19:10:11 -08:00
Torben Giesselmann
8161605f1b
snapshot_group: Fix typo
2023-03-03 19:07:57 -08:00
dependabot[bot]
b78607c9d8
build(deps): bump github.com/klauspost/compress from 1.15.15 to 1.16.0
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.15.15 to 1.16.0.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.15.15...v1.16.0 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-27 02:52:10 +00:00
Niklas Wagner
c1101ede19
Add jq to container image
2023-02-21 14:01:43 +01:00
Michael Eischer
f646406822
Merge pull request #4182 from BenWiederhake/dev-with-cache-description
...
check: Document behavior of --with-cache a bit better
2023-02-19 19:41:38 +01:00
Ben Wiederhake
9888443f5c
check: Document behavior of --with-cache a bit better
...
Also see https://forum.restic.net/t/how-to-speed-up-tiny-incremental-checks/5905
2023-02-19 17:45:47 +01:00
Michael Eischer
cf6dfd6d36
Merge pull request #4212 from greatroar/snapshotfilter
...
cmd, restic: Refactor and fix snapshot filtering
2023-02-19 15:20:18 +01:00
greatroar
a7786c67f1
cmd, restic: Refactor and fix snapshot filtering
...
This turns snapshotFilterOptions from cmd into a restic.SnapshotFilter
type and makes restic.FindFilteredSnapshot and FindFilteredSnapshots
methods on that type. This fixes #4211 by ensuring that hosts and paths
are named struct fields instead of unnamed function arguments in long
lists of such.
Timestamp limits are also included in the new type. To avoid too much
pointer handling, the convention is that time zero means no limit.
That's January 1st, year 1, 00:00 UTC, which is so unlikely a date that
we can sacrifice it for simpler code.
2023-02-19 15:04:25 +01:00
Michael Eischer
15b7d9c80b
Merge pull request #4213 from restic/dependabot/go_modules/golang.org/x/net-0.7.0
...
build(deps): bump golang.org/x/net from 0.6.0 to 0.7.0
2023-02-19 12:42:01 +01:00
dependabot[bot]
ee4128281e
build(deps): bump golang.org/x/net from 0.6.0 to 0.7.0
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases )
- [Commits](https://github.com/golang/net/compare/v0.6.0...v0.7.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-19 11:32:55 +00:00
Michael Eischer
3c8aefa0cb
Merge pull request #4205 from restic/dependabot/go_modules/golang.org/x/oauth2-0.5.0
...
build(deps): bump golang.org/x/oauth2 from 0.4.0 to 0.5.0
2023-02-19 12:31:43 +01:00
aawsome
e2df73b0ac
Update doc/design.rst
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2023-02-17 10:07:53 +01:00
aawsome
f4329a20f6
docs: Corrections/extra information in design.rst
2023-02-16 22:41:30 +01:00
dependabot[bot]
11ebc0c5db
build(deps): bump golang.org/x/oauth2 from 0.4.0 to 0.5.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/golang/oauth2/releases )
- [Commits](https://github.com/golang/oauth2/compare/v0.4.0...v0.5.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-13 02:54:09 +00:00
Michael Eischer
f137be42fe
Merge pull request #4081 from MichaelEischer/backup-group-by
...
Add support for `--group-by` to the backup command
2023-02-11 11:38:13 +01:00
Michael Eischer
0ce182f044
document backup --group-by
2023-02-11 00:55:11 +01:00
Alex Thomson
1b50faf03e
Add changelog
2023-02-10 23:45:20 +01:00
Alex Thomson
4cbbf5d952
Fix scan_finished JSON MessageType
...
Corrected the scan_finished JSON MessageType from status to verbose_status
2023-02-10 23:45:20 +01:00
Michael Eischer
2885db7902
backup: add group-by option
2023-02-10 23:18:14 +01:00
Michael Eischer
acb40d2b94
Refactor group-by to parse options into a struct
2023-02-10 23:18:14 +01:00
Michael Eischer
fa73b50b45
Merge pull request #4198 from MichaelEischer/better-verbose-description
...
Make help for --verbose less confusing
2023-02-09 22:28:18 +01:00
Michael Eischer
2d700c3887
Make help for --verbose less confusing
...
The output is now
```
-v, --verbose be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)
```
instead of
```
-v, --verbose n be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)
```
2023-02-06 22:11:21 +01:00
Michael Eischer
91251f2d57
Merge pull request #4196 from restic/dependabot/github_actions/docker/build-push-action-4
...
build(deps): bump docker/build-push-action from 3 to 4
2023-02-06 21:54:06 +01:00
Michael Eischer
3df4ec7c61
Merge pull request #4195 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.3.1
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.3.0 to 1.3.1
2023-02-06 21:20:33 +01:00
Michael Eischer
b1d3a1a5e3
Merge pull request #4194 from restic/dependabot/go_modules/google.golang.org/api-0.109.0
...
build(deps): bump google.golang.org/api from 0.108.0 to 0.109.0
2023-02-06 21:17:57 +01:00
dependabot[bot]
4f31c2699d
build(deps): bump docker/build-push-action from 3 to 4
...
Bumps [docker/build-push-action](https://github.com/docker/build-push-action ) from 3 to 4.
- [Release notes](https://github.com/docker/build-push-action/releases )
- [Commits](https://github.com/docker/build-push-action/compare/v3...v4 )
---
updated-dependencies:
- dependency-name: docker/build-push-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-06 02:13:17 +00:00
dependabot[bot]
96b1ff5e38
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.3.0 to 1.3.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.3.0...sdk/azcore/v1.3.1 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-06 02:07:34 +00:00
dependabot[bot]
6b5ffce9dc
build(deps): bump google.golang.org/api from 0.108.0 to 0.109.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.108.0 to 0.109.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.108.0...v0.109.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-06 02:06:57 +00:00
Michael Eischer
590eb9efd7
Merge pull request #4180 from MichaelEischer/add-riscv
...
Add linux/riscv64 builds
2023-01-30 22:57:39 +01:00
Michael Eischer
55c4ca66f7
add linux/riscv64 builds
2023-01-30 22:24:12 +01:00
Michael Eischer
56ad761b19
Merge pull request #4177 from greatroar/loadunpacked
...
repository: Reuse buffers in Repository.LoadUnpacked
2023-01-30 22:12:13 +01:00
greatroar
d129baba7a
repository: Reuse buffers in Repository.LoadUnpacked
...
This method had a buffer argument, but that was nil at all call sites.
That's removed, and instead LoadUnpacked now reuses whatever it
allocates inside its retry loop.
2023-01-30 22:01:01 +01:00
Alexander Neumann
febb32b5b4
Set development version for 0.15.1
2023-01-30 20:43:52 +01:00
Alexander Neumann
7d4b7ad9cb
Add version for 0.15.1
2023-01-30 20:43:47 +01:00
Alexander Neumann
a883bb6596
Generate CHANGELOG.md for 0.15.1
2023-01-30 20:43:15 +01:00
Alexander Neumann
91acef90b2
Prepare changelog for 0.15.1
2023-01-30 20:43:15 +01:00
Michael Eischer
b2b7727b31
Merge pull request #4178 from rawtaz/changelogs
...
Polish changelogs
2023-01-27 22:46:46 +01:00
Leo R. Lundgren
0e4c9a5421
Polish changelogs
2023-01-27 22:27:57 +01:00
Michael Eischer
49fa8fe6dd
Merge pull request #4175 from fergus-dall/deadlock-fix
...
Fix deadlock in Lock.Stale
2023-01-26 22:45:13 +01:00
Michael Eischer
12f167ee79
Merge pull request #4167 from aneesh-n/progress-percent-eta-fix
...
ui/backup: Fix percent and eta in backup progress
2023-01-26 22:42:10 +01:00
Michael Eischer
bb018fbc3e
Merge pull request #4163 from MichaelEischer/fix-windows-self-upgrade
...
self-upgrade: Fix handling of `--output` on windows
2023-01-26 22:40:36 +01:00
Michael Eischer
3b24e0ac55
Merge pull request #4154 from MichaelEischer/cleanup-prune-tests
...
deduplicate prune tests a bit
2023-01-26 22:39:15 +01:00
Fergus Dall
04da31af2b
Fix deadlock in Lock.Stale
...
With debug logging enabled this method would take a lock and then
format the lock as a string. Since PR #4022 landed the string
formatting method has also taken the lock, so this deadlocks.
Instead just record the lock ID, as is done elsewhere.
2023-01-27 03:21:37 +11:00
Alexander Neumann
65923e9c26
Merge pull request #4152 from MichaelEischer/invalid-locks
...
Improve handling of invalid locks
2023-01-25 08:19:53 +01:00
Michael Eischer
b903081804
Merge pull request #4169 from restic/dependabot/go_modules/github.com/klauspost/compress-1.15.15
...
build(deps): bump github.com/klauspost/compress from 1.15.14 to 1.15.15
2023-01-23 22:35:52 +01:00
Michael Eischer
beb1e872cc
Merge pull request #4170 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.29.0
...
build(deps): bump cloud.google.com/go/storage from 1.28.1 to 1.29.0
2023-01-23 22:25:09 +01:00
Michael Eischer
db350c0430
Merge pull request #4171 from restic/dependabot/go_modules/google.golang.org/api-0.108.0
...
build(deps): bump google.golang.org/api from 0.107.0 to 0.108.0
2023-01-23 22:23:51 +01:00
dependabot[bot]
716a5dd20d
build(deps): bump google.golang.org/api from 0.107.0 to 0.108.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.107.0 to 0.108.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.107.0...v0.108.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-23 02:23:51 +00:00
dependabot[bot]
dbd07ade98
build(deps): bump cloud.google.com/go/storage from 1.28.1 to 1.29.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.28.1 to 1.29.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/storage/v1.28.1...spanner/v1.29.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-23 02:23:33 +00:00
dependabot[bot]
7adf1e5d37
build(deps): bump github.com/klauspost/compress from 1.15.14 to 1.15.15
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.15.14 to 1.15.15.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.15.14...v1.15.15 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-23 02:23:15 +00:00
Michael Eischer
8f94eb5420
add changelog for less strict lock handling
2023-01-22 15:54:07 +01:00
Michael Eischer
8aaba83719
add changelog for self-upgrade bug on windows
2023-01-22 15:39:42 +01:00
Michael Eischer
e16a6d4c50
self-update: add basic test for extractToFile
2023-01-22 15:39:42 +01:00
Michael Eischer
34e67e3510
self-update: Fix handling of `--output` on windows
...
The code always assumed that the upgrade happens in place. Thus writing
the upgrade to a separate file fails, when trying to remove the file
stored at that location.
2023-01-22 15:39:42 +01:00
Michael Eischer
c527c05590
add changelog for missing backup ETA
2023-01-22 15:32:55 +01:00
Aneesh Nireshwalia
ed23edeb62
ui/backup: Fix percent and eta in backup progress
...
Added missing call to scanFinished=true.
This was causing the percent and eta to never get
printed for backup progress even after the scan was finished.
2023-01-21 13:25:08 -07:00
Michael Eischer
0f398b82e3
Merge pull request #4145 from greatroar/index-encode
...
index: Optimize generatePackList
2023-01-21 01:01:12 +01:00
greatroar
99755c634b
index: Optimize generatePackList
...
name old time/op new time/op delta
EncodeIndex/100-8 1.56ms ± 2% 1.48ms ± 3% -5.37% (p=0.000 n=10+10)
EncodeIndex/1000-8 14.5ms ± 2% 13.1ms ± 2% -9.49% (p=0.000 n=9+10)
EncodeIndex/10000-8 120ms ± 2% 116ms ± 2% -3.58% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
EncodeIndex/100-8 306kB ± 1% 275kB ± 1% -10.28% (p=0.000 n=10+10)
EncodeIndex/1000-8 3.69MB ±11% 2.88MB ± 5% -22.07% (p=0.000 n=10+9)
EncodeIndex/10000-8 35.9MB ±11% 31.9MB ±10% -11.13% (p=0.005 n=10+10)
name old allocs/op new allocs/op delta
EncodeIndex/100-8 3.39k ± 0% 2.39k ± 0% -29.61% (p=0.000 n=10+10)
EncodeIndex/1000-8 32.6k ± 0% 22.9k ± 0% -29.63% (p=0.000 n=10+9)
EncodeIndex/10000-8 326k ± 0% 229k ± 0% -29.71% (p=0.000 n=10+10)
The bulk of the allocation rate improvement comes from just removing the
debug.Log calls: every one of those copied a restic.ID to the heap.
2023-01-14 20:41:07 +01:00
Michael Eischer
f5f13f6648
Merge pull request #4153 from MichaelEischer/fix-quiet-prune
...
prune: Don't show "packs processed" for quiet runs
2023-01-14 20:11:34 +01:00
Michael Eischer
00216d54a1
deduplicate prune tests a bit
2023-01-14 19:05:45 +01:00
Michael Eischer
1f3f68b2c0
prune: Don't show "packs processed" for quiet runs
2023-01-14 18:48:32 +01:00
Michael Eischer
57acc769b4
lock: Ignore empty lock files
...
These may be left behind by backends which do not guarantee atomic
uploads.
2023-01-14 18:15:46 +01:00
Michael Eischer
20ad14e362
lock: add help message how to recover from invalid locks
2023-01-14 18:04:22 +01:00
Michael Eischer
c995b5be52
lock: cleanup error message
...
The error message is now `Fatal: unable to create lock in backend:
[...]` instead of `unable to create lock in backend: Fatal: [...]`.
2023-01-14 17:57:02 +01:00
Michael Eischer
1adf28a2b5
repository: properly return invalid data error in LoadUnpacked
...
The retry backend does not return the original error, if its execution
is interrupted by canceling the context. Thus, we have to manually
ensure that the invalid data error gets returned.
Additionally, use the retry backend for some of the repository tests, as
this is the configuration which will be used by restic.
2023-01-14 17:57:02 +01:00
Michael Eischer
6d9675c323
repository: cleanup error message on invalid data
...
The retry printed the filename twice:
```
Load(<lock/04804cba82>, 0, 0) returned error, retrying after 720.254544ms: load(<lock/04804cba82>): invalid data returned
```
now the warning has changed to
```
Load(<lock/04804cba82>, 0, 0) returned error, retrying after 720.254544ms: invalid data returned
```
2023-01-14 17:57:02 +01:00
Michael Eischer
551b31ce3c
Merge pull request #4146 from MichaelEischer/update-blazer
...
Update blazer to fix `b2_download_file_by_name: 404`
2023-01-14 15:23:32 +01:00
Michael Eischer
ec99507e4c
update blazer
...
This removes a stray warning, that was printed when checking the status
of non-existent files.
2023-01-14 14:56:25 +01:00
Michael Eischer
5f97f534b1
Merge pull request #4150 from restic/dependabot/go_modules/google.golang.org/api-0.107.0
...
build(deps): bump google.golang.org/api from 0.106.0 to 0.107.0
2023-01-14 14:55:15 +01:00
Michael Eischer
ed11bbd0e2
Merge pull request #4151 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.47
...
build(deps): bump github.com/minio/minio-go/v7 from 7.0.46 to 7.0.47
2023-01-14 14:50:01 +01:00
dependabot[bot]
5bb9cb056d
build(deps): bump github.com/minio/minio-go/v7 from 7.0.46 to 7.0.47
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.46 to 7.0.47.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.46...v7.0.47 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-14 13:38:58 +00:00
dependabot[bot]
cd9bd22563
build(deps): bump google.golang.org/api from 0.106.0 to 0.107.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.106.0 to 0.107.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.106.0...v0.107.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-14 13:38:42 +00:00
Michael Eischer
ecc826ef7d
Merge pull request #4149 from MichaelEischer/more-frequent-dependabot
...
CI: let dependabot check for updates weekly
2023-01-14 14:37:30 +01:00
Michael Eischer
fb43cbab49
Merge pull request #4129 from greatroar/cleanup
...
cache: Replace readCloser+LimitedReader by backend.LimitedReadCloser
2023-01-14 12:17:25 +01:00
Michael Eischer
41d31b1e27
Merge pull request #4116 from MichaelEischer/extract-progress-updater
...
ui/progress: Extract progress updater
2023-01-14 12:07:26 +01:00
Michael Eischer
f6ea5c5865
CI: let dependabot check for updates weekly
2023-01-14 12:02:42 +01:00
Michael Eischer
4a7a6b06af
ui/backup: Use progress.Updater for progress updates
2023-01-14 01:20:43 +01:00
Michael Eischer
e499bbe3ae
progress: extract progress updating into Updater struct
...
This allows reusing the code to create periodic progress updates.
2023-01-14 01:13:08 +01:00
Michael Eischer
52682b1c7b
Merge pull request #4111 from MichaelEischer/extract-stdio-wrapper
...
backup: extract StdioWrapper from ProgressPrinters
2023-01-14 01:12:23 +01:00
Michael Eischer
c15b4bceae
backup: extract StdioWrapper from ProgressPrinters
...
The StdioWrapper is not used at all by the ProgressPrinters. It is
called a bit earlier than previously. However, as the password prompt
directly accessed stdin/stdout this doesn't cause problems.
2023-01-14 00:58:13 +01:00
Alexander Neumann
74348be3fa
Set development version for 0.15.0
2023-01-12 20:51:23 +01:00
Alexander Neumann
37d0e1fe58
Add version for 0.15.0
2023-01-12 20:51:19 +01:00
Alexander Neumann
da196aa43e
Update manpages and auto-completion
2023-01-12 20:51:19 +01:00
Alexander Neumann
099774c2aa
Generate CHANGELOG.md for 0.15.0
2023-01-12 20:50:45 +01:00
Alexander Neumann
cd2f53e3f9
Prepare changelog for 0.15.0
2023-01-12 20:50:44 +01:00
Alexander Neumann
0c5a55d1bd
Merge pull request #4142 from restic/update-deps
...
Upgrade dependencies
2023-01-12 20:23:25 +01:00
Michael Eischer
9ddca65f6d
update dependencies
2023-01-11 23:22:10 +01:00
Michael Eischer
06fee601bc
Merge pull request #4141 from restic/doc-no-scan-correction
...
doc: Correct heading level for --no-scan
2023-01-11 22:01:05 +01:00
Leo R. Lundgren
1cb920cc57
doc: Correct heading level for --no-scan
2023-01-11 21:47:38 +01:00
Michael Eischer
8f53ffb921
Merge pull request #4140 from restic/doc-manual
...
doc: Update manual page with --no-scan
2023-01-11 21:42:42 +01:00
Michael Eischer
351cbb4f94
Merge pull request #4139 from restic/doc-no-scan
...
doc: Move and update documentation for --no-scan
2023-01-11 21:41:20 +01:00
Leo R. Lundgren
b8b5508d15
doc: Update manual page with --no-scan
2023-01-11 02:27:43 +01:00
Leo R. Lundgren
c5542ddcd2
doc: Move and update documentation for --no-scan
2023-01-11 00:09:24 +01:00
rawtaz
dffb8e0c14
Merge pull request #4138 from MichaelEischer/doc-sparse
...
doc: add description for restore --sparse
2023-01-10 23:39:51 +01:00
Michael Eischer
b151fa498a
doc: add description for restore --sparse
2023-01-10 23:27:42 +01:00
Michael Eischer
c354b55e62
Merge pull request #4136 from restic/doc-small-files
...
doc: Clarify text about tuning backups for small files
2023-01-08 22:00:09 +01:00
Michael Eischer
375953a001
Merge pull request #4088 from MichaelEischer/doc-cifs-backup-source
...
doc: reading from CIFS can be a problem on linux
2023-01-08 21:56:48 +01:00
Leo R. Lundgren
6306797238
doc: Clarify text about tuning backups for small files
2023-01-08 21:49:55 +01:00
Michael Eischer
ef9164fcbb
doc: reading from CIFS can be a problem on linux
2023-01-08 21:47:34 +01:00
rawtaz
e2bcfd68dd
Merge pull request #4135 from restic/changelogs
...
Polish changelogs
2023-01-08 20:04:24 +01:00
Leo R. Lundgren
33fb351386
Polish changelogs
2023-01-08 19:48:51 +01:00
Michael Eischer
c9840da4f8
Merge pull request #4134 from MichaelEischer/fix-verbose-help-text
...
Correct maximum verbosity level in help message
2023-01-07 22:24:04 +01:00
Michael Eischer
732184a849
Correct maximum verbosity level in help message
...
The maximum for `--verbose=n` is n=2. Internally it is translated into a
scale from 0 to 3. However, the default (without verbose) is 1, thus the
verbosity level can only be increased two times.
2023-01-07 22:02:13 +01:00
Michael Eischer
24178c97e9
Merge pull request #4117 from MichaelEischer/prune-dry-run-help
...
prune: make it clearer when prune is used in dry-run mode
2023-01-04 23:18:53 +01:00
Michael Eischer
7a36306901
forget: Clarify log message for `--dry-run --prune`
2023-01-04 00:44:46 +01:00
Michael Eischer
b404ad4eaa
prune: make it clearer when prune is used in dry-run mode
2023-01-04 00:44:46 +01:00
Michael Eischer
e02a10c58a
Merge pull request #4109 from MichaelEischer/fix-prune-uncompressed-accounting
...
prune: Fix calculation of remaining uncompressed data
2023-01-03 23:28:10 +01:00
Michael Eischer
81dc8c8d13
prune: Fix calculation of remaining uncompressed data
...
Only the repacking of *un*compressed packs reduces the amount of
uncompressed data. Previously the counter even overflowed for fully
compressed repositories.
2023-01-03 22:34:36 +01:00
greatroar
72922a79ed
cache: Replace readCloser+LimitedReader by backend.LimitedReadCloser
2023-01-03 19:03:36 +01:00
Michael Eischer
89a8006578
Merge pull request #4104 from philaris/fix_max_uint32_uid_gid_to_zero
...
in tar dump, convert uid, gid of value -1 to zero
2023-01-02 22:28:28 +01:00
Panagiotis Cheilaris
3b516d4b70
convert uid/gid -1 to 0 only in 32-bit tar dump
...
Only for a 32-bit build of restic, convert a uid or gid value of -1 to 0.
2022-12-30 18:12:12 +01:00
Michael Eischer
0de3b24756
Merge pull request #4110 from MichaelEischer/remove-exitf
...
Remove Exitf function
2022-12-29 12:07:51 +01:00
Michael Eischer
0fbff39ae8
Merge pull request #4108 from MichaelEischer/cleanup-check-output
...
Cleanup check output
2022-12-29 11:59:18 +01:00
Michael Eischer
68b1f30733
Remove Exitf function
...
Commands should use the normal shutdown path. In addition, the Exitf
function was only used by `dump` and `restore` but not any other command
which introduces the risk of inconsistent behavior.
2022-12-28 21:42:38 +01:00
Panagiotis Cheilaris
a86a56cf3b
fix lint issue with function name 'tarId'
...
See https://github.com/golang/lint/issues/89 and
https://github.com/golang/lint/issues/124
2022-12-28 18:46:58 +01:00
Panagiotis Cheilaris
050ed616ae
be more explicit with uid or gid of value -1
2022-12-28 18:44:36 +01:00
Michael Eischer
8430399fce
check: Partially fix garbled output
...
When reporting an error for a tree, the output message can overlap with
the progress bar output, e.g. `error for tree e91ef6fb:napshots`.
The fix only applies for this specific message and does not work on
Windows.
2022-12-28 17:47:27 +01:00
Michael Eischer
aea96b7d86
check: Slightly improve help message
...
If a repository has both pack/index related warnings and errors, then
the help message is quite misleading. Reword it slightly to be more
clear.
2022-12-28 17:46:06 +01:00
Michael Eischer
bcae28afb4
Merge pull request #4100 from klemensn/tag-self-update
...
Reinstate selfupdate tag to make builds without self-update work
2022-12-28 15:49:11 +01:00
Panagiotis Cheilaris
10fa5cde0a
in tar dump, convert uid, gid of value -1 to zero
2022-12-27 16:36:04 +01:00
Klemens Nanni
61e7386384
Bugfix: Make distribution package builds without self-update work
2022-12-26 21:52:24 +04:00
Klemens Nanni
94f6e7d4a6
Reinstate selfupdate tag to make builds without self-update work
...
Revert what seems to be a typo introduced as part of the fix for #2041
in 2018 7d0f2eaf24
.
`xbuild` does not look like a go build/tag keyword to me, I failed to
find documentation for it and using `go install -tags '!selfupdate' ...`
has no effect, i.e. self-update code is still compiled.
`+build` however works; updating the OpenBSD port/binary package
security/restic to apply this PR works as expected:
```
$ restic help | grep self
$ restic self-update
unknown command "self-update" for "restic"
```
(Using `go:build` now as per restic's style and gofmt.)
Previously, using `restic-0.14.0p1` on OpenBSD/amd64 7.2-current would
check for a newer version and probably attempt replacing the system wide
root-owned executable (on a read-only filesystem) as unprivileged user:
```
$ restic version
restic 0.14.0 compiled with go1.19.2 on openbsd/amd64
$ restic help | grep self
self-update Update the restic binary
$ restic self-update
writing restic to /usr/local/bin/restic
find latest release of restic at GitHub
restic is up to date
```
(It never tried to actually write besaid path; doing so would fail, so
the current message can be considered misleading.)
2022-12-26 21:46:22 +04:00
Michael Eischer
90fb6f70b4
Merge pull request #4089 from greatroar/errors
...
Clean up error handling further
2022-12-24 10:41:56 +01:00
Michael Eischer
29b8500254
Merge pull request #4090 from restic/upgrade-dependencies
...
Upgrade dependencies
2022-12-23 22:34:39 +01:00
Michael Eischer
705cabb304
Merge pull request #3981 from MichaelEischer/prune-uncompressed-stats
...
prune: report how much data must be repacked to compressed the repo
2022-12-23 22:34:04 +01:00
Michael Eischer
a6f3ae5790
Merge pull request #4094 from googol42/master
...
remove duplicated init
2022-12-23 22:33:01 +01:00
Andreas Dominik Preikschat
ea37240597
remove duplicated init
...
the documentation contained the `init` command twice
2022-12-20 17:24:56 +01:00
Michael Eischer
bd2f6aaac3
azure: downgrade azblob dependency due to build breakages on Solaris
2022-12-17 23:35:07 +01:00
Michael Eischer
583372956b
Upgrade dependencies
...
Nothing special has changed.
2022-12-17 15:23:11 +01:00
greatroar
1678392a6d
checker: Make ErrLegacyLayout a value, not a type
2022-12-17 09:41:07 +01:00
greatroar
d9002f050e
backend: Don't Wrap errors from url.Parse
...
The messages from url.Error.Error already start with the word "parse".
2022-12-17 09:41:07 +01:00
greatroar
b150dd0235
all: Replace some errors.Wrap calls by errors.WithStack
...
Mostly changed the ones that repeat the name of a system call, which is
already contained in os.PathError.Op. internal/fs.Reader had to be
changed to actually return such errors.
2022-12-17 09:41:07 +01:00
Michael Eischer
cccc17e4e9
Merge pull request #4086 from blackpiglet/modify_access_denied_code
...
Fix: change error code in function isAccessDenied to AccessDenied
2022-12-16 21:55:52 +01:00
Michael Eischer
2723159ed4
Merge pull request #3931 from kjetilho/feature/optional_scanner
...
add --no-scan to backup command
2022-12-16 21:42:22 +01:00
Xun Jiang/Bruce Jiang
ecc62c8be2
Update changelog/unreleased/issue-4085
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-12-16 21:41:16 +01:00
Xun Jiang
cc5325d22b
Fix: change error code in function isAccessDenied to AccessDenied
...
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-12-16 21:41:16 +01:00
Michael Eischer
da0e45cf40
Merge pull request #4083 from greatroar/cleanup
...
repository: Remove empty cleanup functions in tests
2022-12-16 21:39:30 +01:00
Kjetil Torgrim Homme
14aa6f2a00
add --disable-scanner to backup command
...
The scanner process has only cosmetic effect for the progress printer,
and can be disabled without impacting functionality when the user does
not need an estimate of completion.
In many cases the scanner process can provide beneficial priming of
the file system cache, so as general advice it should not be disabled.
However, tests have shown that backup of NFS and fuse based filesystems,
where stat(2) is relatively expensive, can be significantly faster
without the scanner.
2022-12-16 21:29:59 +01:00
Michael Eischer
7bdb985dde
Merge pull request #4079 from MichaelEischer/rewrite-set-original
...
rewrite: Always set the Original field in a rewritten snapshot
2022-12-13 22:56:20 +01:00
Michael Eischer
1bfe98bdc0
Merge pull request #2398 from DanielG/b2-hide-file
...
b2: Fallback to b2_hide_file when delete returns unauthorized
2022-12-13 22:52:23 +01:00
Michael Eischer
1c071a462e
Merge pull request #4084 from ekarlso/azure-stat-fix
...
fix: Make create not error out when ContainerNotFound
2022-12-13 22:49:05 +01:00
Michael Eischer
25d22d5241
Merge pull request #4082 from MichaelEischer/unbuffered-logger-for-testing
...
Don't buffer the golang `log` package output when running tests
2022-12-13 22:45:50 +01:00
Endre Karlson
7dd33c0ecc
azure: Make create not error out when ContainerNotFound
2022-12-11 22:57:23 +01:00
greatroar
c0b5ec55ab
repository: Remove empty cleanup functions in tests
...
TestRepository and its variants always returned no-op cleanup functions.
If they ever do need to do cleanup, using testing.T.Cleanup is easier
than passing these functions around.
2022-12-11 11:06:25 +01:00
Michael Eischer
2e3d4640be
Don't buffer the golang log output when running tests
2022-12-10 16:08:27 +01:00
Michael Eischer
38b2e9b42c
rewrite: Always set the Original field in a rewritten snapshot
...
The Original field is meant to remember the original snapshot id if e.g.
changing its tags. It was only set by the `rewrite` command if it was
not set previously. However, a rewritten snapshot is potentially rather
different from the original snapshot. Thus just always set the Original
field. This also makes it easier to later on detect and potentially
remove the original snapshots.
2022-12-10 12:47:00 +01:00
Michael Eischer
049a105ba5
Merge pull request #4077 from greatroar/cleanup
...
test: Use testing.T.Cleanup to remove tempdirs
2022-12-09 22:17:46 +01:00
Michael Eischer
4b98b5562d
Merge pull request #4075 from greatroar/sftp-enospc
...
sftp: Fix ENOSPC check
2022-12-09 22:00:13 +01:00
greatroar
f90bf84ba7
test: Use testing.T.Cleanup to remove tempdirs
2022-12-09 14:23:55 +01:00
greatroar
83d23b3ae8
Changelog for ENOSPC handling bug
2022-12-09 08:50:30 +01:00
Michael Eischer
eae7366563
Merge pull request #4028 from ekarlso/use-az-blob-sdk
...
Switch to azblob sdk
2022-12-07 21:58:03 +01:00
Endre Karlson
25648e2501
azure: Switch to azblob sdk
2022-12-07 21:46:07 +01:00
greatroar
62520bb7b4
sftp: Fix ENOSPC check
...
We now check for space that is not reserved for the root user on the
remote, and the check is no longer in a defer block because it wouldn't
fire. Some change in the surrounding code may have led the deferred
function to capture the wrong err variable.
Fixes #3336 .
2022-12-07 21:06:46 +01:00
rawtaz
4ba31df08f
Merge pull request #4074 from greatroar/lobaro-docker
...
doc: Remove ref to Lobaro's Docker image
2022-12-04 18:00:08 +01:00
greatroar
5efcbe143c
doc: Remove ref to Lobaro's Docker image
...
It hasn't been updated for a while and has restic 0.12.0. Fixes #4002 .
2022-12-04 16:20:42 +01:00
Michael Eischer
0df585dd99
Merge pull request #4066 from sedlund/fix#4033
...
fix#4033 cmd: copy no longer lists skipped existing snapshots by default
2022-12-03 19:22:11 +01:00
Michael Eischer
223da7344e
Merge pull request #4070 from restic/fix-cloud-tests
...
Fix cloud tests
2022-12-03 19:21:25 +01:00
Michael Eischer
2b67862420
backend/test: check that IsNotExist actually works
2022-12-03 18:56:55 +01:00
Michael Eischer
2f934f5803
gs: check against the correct error in IsNotExist
2022-12-03 18:49:54 +01:00
Michael Eischer
04d101fa94
gs/s3: remove useless os.IsNotExist check
2022-12-03 18:49:54 +01:00
Michael Eischer
579cd6dc64
azure: fix totally broken IsNotExist
2022-12-03 18:49:54 +01:00
Michael Eischer
3ebdadc58f
Merge pull request #4069 from greatroar/cleanup
...
cache, prune, restic: Cleanup
2022-12-03 17:50:52 +01:00
Michael Eischer
bc8b2455b9
Merge pull request #4064 from MichaelEischer/flaky-abort-early-on-error
...
archiver: Fix flaky TestArchiverAbortEarlyOnError
2022-12-03 17:43:44 +01:00
Michael Eischer
60c6a09324
Merge pull request #4065 from MichaelEischer/flaky-rclone-failed-start
...
rclone: treat "file already closed" as command startup error
2022-12-03 17:42:56 +01:00
Michael Eischer
8bf6b2b80d
Merge pull request #4067 from MichaelEischer/remove-backend-test-method
...
Remove `Test()` method from Backend
2022-12-03 17:40:55 +01:00
Michael Eischer
78ea69082a
Merge pull request #4068 from MichaelEischer/debug-lock-refresh-test
...
Add more debug logging to `TestLockSuccessfulRefresh`
2022-12-03 17:38:21 +01:00
Scott Edlund
cbe73ace3f
Update changelog/unreleased/issue-4033
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2022-12-03 20:07:37 +08:00
greatroar
63bed34608
restic: Clean up restic.IDs type
...
IDs.Less can be rewritten as
string(list[i][:]) < string(list[j][:])
Note that this does not copy the ID's.
The Uniq method was no longer used.
The String method has been reimplemented without first copying into a
separate slice of a custom type.
2022-12-03 12:38:20 +01:00
greatroar
0c749dd358
prune: Pass fewer options around
2022-12-03 12:14:04 +01:00
greatroar
d45a2475e1
cache: Rewrite unnecessary if-else
2022-12-03 12:13:54 +01:00
Michael Eischer
6b5d6b9f2c
Add more debug logging to TestLockSuccessfulRefresh
...
The test fails from time to time. Add some more logging to hopefully get
an idea where things go wrong.
2022-12-03 12:05:38 +01:00
Michael Eischer
648edeca40
retry: Do not retry Stat() if file does not exist
...
In non test/debug code, Stat() is used exclusively to check whether a
file exists. Thus, do not retry if a file is reported as not existing.
2022-12-03 11:42:48 +01:00
Michael Eischer
40ac678252
backend: remove Test method
...
The Test method was only used in exactly one place, namely when trying
to create a new repository it was used to check whether a config file
already exists.
Use a combination of Stat() and IsNotExist() instead.
2022-12-03 11:28:10 +01:00
sedlund
06ee0339aa
fix#4033 cmd: copy no longer lists skipped existing snapshots by default
2022-12-03 09:55:39 +08:00
Michael Eischer
57d8eedb88
Merge pull request #4020 from greatroar/fuse-inode
...
fuse: Better inode generation
2022-12-02 22:28:15 +01:00
Michael Eischer
ca1803cacb
Merge pull request #4063 from MichaelEischer/replace-ioutil-usage
...
Replace ioutil usage
2022-12-02 21:49:40 +01:00
Michael Eischer
0af89a5738
Merge pull request #3132 from metalsp0rk/init-json
...
Init command JSON output
2022-12-02 21:49:22 +01:00
Michael Eischer
364a396fd6
init: use standard name `message_type` to distinguish JSON messages
2022-12-02 21:33:03 +01:00
Michael Eischer
9a9f559806
init: cleanup json print code
2022-12-02 21:33:03 +01:00
Kyle Brennan
933c9af328
create changelog entry for issue-3124 and pull-3132
2022-12-02 21:32:30 +01:00
Kyle Brennan
a6ae79b39e
support json output for init command
2022-12-02 21:32:30 +01:00
Michael Eischer
f3d964a8c1
rclone: treat "file already closed" as command startup error
...
Since #3940 the rclone backend returns the commands exit code if it
fails to start. The list of expected errors was missing the "file
already closed"-error which can occur if the http test request first
learns about the closed pipe to rclone before noticing the canceled
context.
Go internally makes sure that a file descriptor is unusable once it was
closed, thus this cannot have unintended side effects (like accidentally
reading from the wrong file due to a reused file descriptor).
2022-12-02 20:46:02 +01:00
Michael Eischer
a9972dbe7d
archiver: Fix flaky TestArchiverAbortEarlyOnError
...
Saving the blobs of a file by now happens asynchronously to the
processing in the FileSaver. Thus we have to account for the blobs
queued for saving.
2022-12-02 20:07:34 +01:00
Michael Eischer
f755233210
Replace usages of ioutil.ReadDir
...
This changes the return type to []fs.DirEntry. However, as we only use
the filenames anyways, this doesn't make a difference.
2022-12-02 19:54:27 +01:00
Michael Eischer
fa20a78bb6
Merge pull request #4056 from greatroar/cleanup
...
backend, fs, options: Minor cleanup
2022-12-02 19:44:54 +01:00
Michael Eischer
ff7ef5007e
Replace most usages of ioutil with the underlying function
...
The ioutil functions are deprecated since Go 1.17 and only wrap another
library function. Thus directly call the underlying function.
This commit only mechanically replaces the function calls.
2022-12-02 19:36:43 +01:00
greatroar
65612d797c
backend, options: Prefer strings.Cut to SplitN
...
Also realigned the various "split into host🪣 prefix"
implementations.
2022-12-02 19:19:14 +01:00
Michael Eischer
2d5e28e777
Merge pull request #4059 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.45
...
build(deps): bump github.com/minio/minio-go/v7 from 7.0.44 to 7.0.45
2022-12-01 21:20:57 +01:00
dependabot[bot]
4fefa2ade2
build(deps): bump github.com/minio/minio-go/v7 from 7.0.44 to 7.0.45
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.44 to 7.0.45.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.44...v7.0.45 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-01 01:03:42 +00:00
Michael Eischer
3c5d1eabe9
Merge pull request #4051 from restic/dependabot/go_modules/github.com/klauspost/compress-1.15.12
...
build(deps): bump github.com/klauspost/compress from 1.15.9 to 1.15.12
2022-11-28 21:47:55 +01:00
Michael Eischer
bec391ee26
Merge pull request #4053 from greatroar/xattr
...
Upgrade pkg/xattr to version with Solaris FIFO fix
2022-11-28 21:06:19 +01:00
greatroar
daafcaf380
Upgrade pkg/xattr to version with Solaris FIFO fix
...
This version doesn't have a release tag yet, but it's 0.4.9 + one patch.
Fixes #4003 .
2022-11-28 20:43:51 +01:00
Alexander Neumann
1d7e7fcd6b
Merge pull request #4049 from MichaelEischer/fix-rewrite-docs
...
rewrite: fix link anchors in documentation
2022-11-28 19:39:38 +01:00
dependabot[bot]
57d59c71e3
build(deps): bump github.com/klauspost/compress from 1.15.9 to 1.15.12
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.15.9 to 1.15.12.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.15.9...v1.15.12 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-28 18:39:35 +00:00
Alexander Neumann
bb83c78ee5
Merge pull request #4047 from MichaelEischer/clean-ci-configuration
...
Cleanup CI configuration
2022-11-28 19:38:33 +01:00
greatroar
60aa87bbab
fs: Remove explicit type check in extendedStat
...
Without comma-ok, the runtime inserts the same check with a similar
enough panic message:
interface conversion: interface {} is nil, not *syscall.Stat_t
2022-11-27 19:58:06 +01:00
Michael Eischer
34609bca0e
Merge pull request #4050 from greatroar/lruv2
...
bloblru: Upgrade to hashicorp/golang-lru/v2
2022-11-27 17:37:14 +01:00
greatroar
e5d597fd22
bloblru: Upgrade to hashicorp/golang-lru/v2
...
The new genericized LRU cache no longer needs to have the IDs separately
allocated:
name old time/op new time/op delta
Add-8 494ns ± 2% 388ns ± 2% -21.46% (p=0.000 n=10+9)
name old alloc/op new alloc/op delta
Add-8 176B ± 0% 152B ± 0% -13.64% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Add-8 5.00 ± 0% 3.00 ± 0% -40.00% (p=0.000 n=10+10)
2022-11-27 17:18:13 +01:00
Michael Eischer
0eddc89e98
doc: design.rst: Fix highlighting for index snippet
...
JSON does not support comments. As JSON is a subset of Javascript, use
the latter instead.
2022-11-27 17:01:27 +01:00
Michael Eischer
41b0f1d43a
doc: fix link to amazon s3 section
2022-11-27 17:01:22 +01:00
Michael Eischer
6a793db9ca
rewrite: fix link anchors in documentation
2022-11-27 16:38:10 +01:00
Michael Eischer
05cebc1c4b
Merge pull request #4044 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.28.0
...
build(deps): bump cloud.google.com/go/storage from 1.25.0 to 1.28.0
2022-11-27 15:23:06 +01:00
Michael Eischer
ce39727846
Merge pull request #4036 from restic/dependabot/go_modules/github.com/pkg/profile-1.7.0
...
build(deps): bump github.com/pkg/profile from 1.6.0 to 1.7.0
2022-11-27 15:22:16 +01:00
Michael Eischer
9aa06ce959
CI: remove option to configure command used to install go tools
...
With the minimum required go version of 1.18, we always use `go
install`.
2022-11-27 15:07:29 +01:00
Michael Eischer
5968971313
CI: remove dependabot ignore for bazil.org/fuse
...
We've switched to a fork of the original library, thus the ignore is no
longer necessary.
2022-11-27 15:06:30 +01:00
dependabot[bot]
95374767de
build(deps): bump github.com/pkg/profile from 1.6.0 to 1.7.0
...
Bumps [github.com/pkg/profile](https://github.com/pkg/profile ) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/pkg/profile/releases )
- [Commits](https://github.com/pkg/profile/compare/v1.6.0...v1.7.0 )
---
updated-dependencies:
- dependency-name: github.com/pkg/profile
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-27 14:02:32 +00:00
dependabot[bot]
c100a62ebf
build(deps): bump cloud.google.com/go/storage from 1.25.0 to 1.28.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.25.0 to 1.28.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.25.0...spanner/v1.28.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-27 14:01:36 +00:00
Michael Eischer
c41a1b66e1
Merge pull request #4037 from restic/dependabot/go_modules/google.golang.org/api-0.103.0
...
build(deps): bump google.golang.org/api from 0.93.0 to 0.103.0
2022-11-27 15:00:16 +01:00
dependabot[bot]
705aed0ecb
build(deps): bump google.golang.org/api from 0.93.0 to 0.103.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.93.0 to 0.103.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.93.0...v0.103.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-27 13:45:08 +00:00
Michael Eischer
28d6de648c
Merge pull request #4040 from restic/dependabot/go_modules/github.com/spf13/cobra-1.6.1
...
build(deps): bump github.com/spf13/cobra from 1.5.0 to 1.6.1
2022-11-27 14:44:25 +01:00
Michael Eischer
bb40b55d1c
Merge pull request #4038 from restic/dependabot/go_modules/github.com/cenkalti/backoff/v4-4.2.0
...
build(deps): bump github.com/cenkalti/backoff/v4 from 4.1.3 to 4.2.0
2022-11-27 14:13:39 +01:00
dependabot[bot]
a24c1e99a6
build(deps): bump github.com/cenkalti/backoff/v4 from 4.1.3 to 4.2.0
...
Bumps [github.com/cenkalti/backoff/v4](https://github.com/cenkalti/backoff ) from 4.1.3 to 4.2.0.
- [Release notes](https://github.com/cenkalti/backoff/releases )
- [Commits](https://github.com/cenkalti/backoff/compare/v4.1.3...v4.2.0 )
---
updated-dependencies:
- dependency-name: github.com/cenkalti/backoff/v4
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-27 12:57:41 +00:00
dependabot[bot]
fd56ead4a8
build(deps): bump github.com/spf13/cobra from 1.5.0 to 1.6.1
...
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra ) from 1.5.0 to 1.6.1.
- [Release notes](https://github.com/spf13/cobra/releases )
- [Commits](https://github.com/spf13/cobra/compare/v1.5.0...v1.6.1 )
---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-27 12:57:23 +00:00
Michael Eischer
cc679c6494
Merge pull request #4041 from MichaelEischer/require-go-1.18
...
Require go 1.18
2022-11-27 13:56:43 +01:00
greatroar
c9c7671c58
fuse: Clean up inode generation
2022-11-27 13:53:42 +01:00
Michael Eischer
530f129a39
rest: remove workaround for content-length handling bug
2022-11-27 13:18:44 +01:00
Michael Eischer
8ad231bcad
bump version numbers in instructions to reproduce binaries
2022-11-27 13:18:44 +01:00
Michael Eischer
a1eb923876
remove no longer necessary conditional compiles
2022-11-27 13:18:44 +01:00
Michael Eischer
bcdfc2a8ea
CI: allow dependabot update of oauth2
...
Our minimum go version is new enough to allow updating the library.
2022-11-27 13:18:44 +01:00
Michael Eischer
686b0b2a3e
update the minimum required go version to 1.18
2022-11-27 13:18:43 +01:00
Michael Eischer
69a2e81bd3
Merge pull request #4039 from restic/dependabot/go_modules/github.com/google/go-cmp-0.5.9
...
build(deps): bump github.com/google/go-cmp from 0.5.8 to 0.5.9
2022-11-26 17:39:27 +01:00
dependabot[bot]
278e93f738
build(deps): bump github.com/google/go-cmp from 0.5.8 to 0.5.9
...
Bumps [github.com/google/go-cmp](https://github.com/google/go-cmp ) from 0.5.8 to 0.5.9.
- [Release notes](https://github.com/google/go-cmp/releases )
- [Commits](https://github.com/google/go-cmp/compare/v0.5.8...v0.5.9 )
---
updated-dependencies:
- dependency-name: github.com/google/go-cmp
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-26 11:35:05 +00:00
Michael Eischer
747d2ecd7b
Merge pull request #4042 from restic/skip-cloud-test-for-dependabot
...
CI: skip cloud tests for dependabot pull requests
2022-11-26 12:34:16 +01:00
Michael Eischer
98c6ca9d8f
CI: skip cloud tests for dependabot pull requests
2022-11-26 12:23:55 +01:00
Michael Eischer
9113b2620f
Merge pull request #4024 from MichaelEischer/macos-fuse
...
mount: switch to anacrolix fork of bazil/fuse
2022-11-26 12:15:14 +01:00
Michael Eischer
f115d64634
Merge pull request #4022 from MichaelEischer/race-checker
...
CI: Run the golang race checker
2022-11-26 12:13:50 +01:00
Michael Eischer
923c06cea0
Merge pull request #4025 from MichaelEischer/update-minio
...
Update minio library to add `credential_process` support
2022-11-25 23:21:57 +01:00
Michael Eischer
f4d3ed77c4
update minio library
2022-11-25 22:36:21 +01:00
greatroar
189e0fe5a9
fuse: Better inode generation
...
Hard links to the same file now get the same inode within the FUSE
mount. Also, inode generation is faster and, more importantly, no longer
allocates.
Benchmarked on Linux/amd64. Old means the benchmark with
sink = fs.GenerateDynamicInode(1, sub.node.Name)
instead of calling inodeFromNode. Results:
name old time/op new time/op delta
Inode/no_hard_links-8 137ns ± 4% 34ns ± 1% -75.20% (p=0.000 n=10+10)
Inode/hard_link-8 33.6ns ± 1% 9.5ns ± 0% -71.82% (p=0.000 n=9+8)
name old alloc/op new alloc/op delta
Inode/no_hard_links-8 48.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10)
Inode/hard_link-8 0.00B 0.00B ~ (all equal)
name old allocs/op new allocs/op delta
Inode/no_hard_links-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
Inode/hard_link-8 0.00 0.00 ~ (all equal)
2022-11-16 08:35:01 +01:00
Michael Eischer
32ffcd86a2
Merge pull request #3993 from MichaelEischer/backup-json-full-snapshot-id
...
backup: print full snapshot id in JSON summary
2022-11-12 20:42:35 +01:00
Michael Eischer
f032a9d0ad
prune: report how much data must be repacked to compressed the repo
...
prune now reports the remaining size of pack files containing
uncompressed blobs. The displayed value is suitable for use with `--max-repack-size`.
2022-11-12 20:20:23 +01:00
Michael Eischer
66818a8f98
Merge pull request #3980 from MichaelEischer/prune-compression-stats
...
prune: Correctly count used/duplicate blobs for partially compressed repos
2022-11-12 20:06:56 +01:00
Michael Eischer
4b5234924b
Merge pull request #2875 from fgma/issue2699
...
issue2699: restore symlinks on windows when run as admin user
2022-11-12 20:06:45 +01:00
Michael Eischer
726a1969cd
Merge pull request #2731 from dionorgua/rewrite-snapshot
...
Implement 'rewrite' command to exclude files from existing snapshots
2022-11-12 20:06:35 +01:00
Michael Eischer
bb0fa76c06
Cleanup exclude pattern collection
2022-11-12 19:55:22 +01:00
Michael Eischer
537cfe2e4c
rewrite: Fix check that an exclude pattern was passed
...
The old check did not consider files containing case insensitive
excludes. The check is now implemented as a function of the
excludePatternOptions struct to improve cohesion.
2022-11-12 19:55:22 +01:00
Leo R. Lundgren
f175da2756
rewrite: Polish documentation
2022-11-12 19:55:22 +01:00
Leo R. Lundgren
f86ef4d3dd
rewrite: Polish code and add missing messages
2022-11-12 19:55:22 +01:00
Leo R. Lundgren
c15bedccc0
rewrite: Revert unrelated documentation change
2022-11-12 19:55:22 +01:00
Michael Eischer
f88acd4503
rewrite: Fail if a tree contains an unknown field
...
In principle, the JSON format of Tree objects is extensible without
requiring a format change. In order to not loose information just play
it safe and reject rewriting trees for which we could loose data.
2022-11-12 19:55:22 +01:00
Michael Eischer
11b8c3a158
rewrite: add documentation
2022-11-12 19:55:22 +01:00
Michael Eischer
ec0c91e233
rewrite: Add tests for further ways to use the command
2022-11-12 19:55:22 +01:00
Michael Eischer
0224e276ec
walker: Add tests for FilterTree
2022-11-12 19:55:22 +01:00
Michael Eischer
73f54cc5ea
rewrite: rename --inplace to --forget
2022-11-12 19:55:22 +01:00
Michael Eischer
a47d9a1c40
rewrite: use unified snapshot filter options
2022-11-12 19:55:22 +01:00
Michael Eischer
b044649118
rewrite: add minimal test
2022-11-12 19:55:22 +01:00
Michael Eischer
375a3db64d
rewrite: non-exclusive lock if snapshots are only added
2022-11-12 19:55:22 +01:00
Michael Eischer
327f418a9c
rewrite: cleanup err handling and output
2022-11-12 19:55:22 +01:00
Michael Eischer
ad14d6e4ac
rewrite: use SelectByName like in the backup command
2022-11-12 19:55:22 +01:00
Michael Eischer
7ebaf6e899
rewrite: start repository uploader goroutines
2022-11-12 19:55:22 +01:00
Michael Eischer
559acea0d8
unify exclude pattern options
2022-11-12 19:55:22 +01:00
Michael Eischer
4cace1ffe9
unify exclude patterns with backup command
2022-11-12 19:55:22 +01:00
Michael Eischer
2b69a1c53b
rewrite: filter all snapshots if none are specified
2022-11-12 19:55:22 +01:00
Michael Eischer
f6339b88af
rewrite: extract tree filtering
2022-11-12 19:55:22 +01:00
Michael Eischer
c0f7ba2388
rewrite: simplify dryrun
2022-11-12 19:55:22 +01:00
Michael Eischer
4d6ab83019
rewrite: use treejsonbuilder
2022-11-12 19:55:22 +01:00
Michael Eischer
82592b88b5
rewrite: address most review comments
2022-11-12 19:55:22 +01:00
Michael Eischer
b922774343
rewrite: fix compilation
2022-11-12 19:55:22 +01:00
Dmitry Nezhevenko
dc29709742
Implement 'rewrite' command to exclude files from existing snapshots
2022-11-12 19:55:22 +01:00
Michael Eischer
220eaee76b
mount: switch to anacrolix fork of bazil/fuse
...
The anacrolix fork contains the latest changes from bazil/fuse and
additionally provides support for recent versions of macFUSE.
2022-11-12 19:22:31 +01:00
Michael Eischer
6fa45d0d39
Merge pull request #4011 from greatroar/backup-stdin-password
...
cmd: Don't read password from stdin for backup --stdin
2022-11-12 19:18:56 +01:00
Michael Eischer
bbd180ae21
Merge pull request #4017 from Rajpratik71/Rajpratik71-patch-1
...
feat: dependabot workflow automation for updating dependency
2022-11-12 15:48:48 +01:00
Pratik Raj
bef1064b8e
chore: ignore upgrade for 'bazil/fuse' and 'golang.org/x/oauth2'
2022-11-12 19:39:16 +05:30
Michael Eischer
7b4fe7bad5
Merge pull request #4021 from greatroar/mac-fsync
...
backend/local: Ignore ENOTTY for fsync on Mac
2022-11-11 23:10:37 +01:00
greatroar
348e966daa
backend/local: Ignore ENOTTY for fsync on Mac
...
Fixes #4016 .
2022-11-11 22:51:51 +01:00
Michael Eischer
0e5fe4c6ab
CI: run golang race checker
2022-11-11 22:15:22 +01:00
Michael Eischer
13fbc96ed3
lock: Synchronize `Refresh()` and `Stale()`
...
The lock test creates a lock and checks that it is not stale. However,
it is possible that the lock is refreshed concurrently, which updates
the lock timestamp. Checking the timestamp in `Stale()` without
synchronization results in a data race. Thus add a lock to prevent
concurrent accesses.
2022-11-11 21:52:53 +01:00
Michael Eischer
e1ba7ab684
lock: Don't copy the lock when checking for process existence
...
The lock test creates a lock and checks that it is not stale. This also
tests whether the corresponding process still exists. However, it is
possible that the lock is refreshed concurrently, which updates the lock
timestamp. Calling `processExists()` with a value receiver, however,
creates an unsynchronized copy of this field. Thus call the method using
a pointer receiver.
2022-11-11 21:45:55 +01:00
Michael Eischer
dc060356c2
mount: only start next test after mount command cleanup is complete
...
The test did not wait for the mount command to fully shutdown all
running goroutines. This caused the go race detector to report a data
race related to lock refreshes.
==================
WARNING: DATA RACE
Write at 0x0000021bdfdb by goroutine 667:
github.com/restic/restic/internal/backend/retry.TestFastRetries()
/restic/restic/internal/backend/retry/testing.go:7 +0x18f
github.com/restic/restic/cmd/restic.withTestEnvironment()
/restic/restic/cmd/restic/integration_helpers_test.go:175 +0x183
github.com/restic/restic/cmd/restic.TestMountSameTimestamps()
/restic/restic/cmd/restic/integration_fuse_test.go:202 +0xac
testing.tRunner()
/usr/lib/go/src/testing/testing.go:1446 +0x216
testing.(*T).Run.func1()
/usr/lib/go/src/testing/testing.go:1493 +0x47
Previous read at 0x0000021bdfdb by goroutine 609:
github.com/restic/restic/internal/backend/retry.(*Backend).retry()
/restic/restic/internal/backend/retry/backend_retry.go:72 +0x9e
github.com/restic/restic/internal/backend/retry.(*Backend).Remove()
/restic/restic/internal/backend/retry/backend_retry.go:149 +0x17d
github.com/restic/restic/internal/cache.(*Backend).Remove()
/restic/restic/internal/cache/backend.go:38 +0x11d
github.com/restic/restic/internal/restic.(*Lock).Unlock()
/restic/restic/internal/restic/lock.go:190 +0x249
github.com/restic/restic/cmd/restic.refreshLocks.func1()
/restic/restic/cmd/restic/lock.go:86 +0xae
runtime.deferreturn()
/usr/lib/go/src/runtime/panic.go:476 +0x32
github.com/restic/restic/cmd/restic.lockRepository.func2()
/restic/restic/cmd/restic/lock.go:61 +0x71
[...]
Goroutine 609 (finished) created at:
github.com/restic/restic/cmd/restic.lockRepository()
/restic/restic/cmd/restic/lock.go:61 +0x488
github.com/restic/restic/cmd/restic.lockRepo()
/restic/restic/cmd/restic/lock.go:25 +0x219
github.com/restic/restic/cmd/restic.runMount()
/restic/restic/cmd/restic/cmd_mount.go:126 +0x1f8
github.com/restic/restic/cmd/restic.testRunMount()
/restic/restic/cmd/restic/integration_fuse_test.go:61 +0x1ce
github.com/restic/restic/cmd/restic.checkSnapshots.func1()
/restic/restic/cmd/restic/integration_fuse_test.go:90 +0x124
==================
2022-11-11 21:43:01 +01:00
Michael Eischer
32c9667990
Merge pull request #4019 from MichaelEischer/fix-file-saver-race
...
archiver: Fix race condition resulting in files containing null IDs
2022-11-11 20:52:33 +01:00
Michael Eischer
d268552a0a
Merge pull request #4014 from MichaelEischer/fix-debug-examine
...
debug: fix crash in `debug examine --reupload-blobs`
2022-11-10 20:37:32 +01:00
Michael Eischer
5756c96c9f
archiver: Fix race condition resulting in files containing null IDs
...
In some rare cases files could be created which contain null IDs (all
zero) in their content list. This was caused by a race condition between
growing the `Content` slice and inserting the blob IDs into it. In some
cases the blob ID was written to the old slice, which a short time
afterwards was replaced with a larger copy, that did not yet contain the
blob ID.
2022-11-10 20:19:37 +01:00
Pratik Raj
df614fff26
feat: dependabot workflow automation for updating dependency
...
Signed-off-by: Pratik Raj <Rajpratik71@gmail.com>
2022-11-10 16:02:03 +05:30
Michael Eischer
11a4bb051e
debug: fix crash in `debug examine --reupload-blobs`
2022-11-09 22:13:17 +01:00
Michael Eischer
5f9ac2b165
Merge pull request #4010 from MichaelEischer/file-saver-sanity-check
...
archiver: Check that saved file does not have null IDs in content
2022-11-08 23:07:32 +01:00
Michael Eischer
b1d1202b1d
archiver: Check that saved file does not have null IDs in content
...
Null IDs in the file content indicate that something went wrong. Thus
fails before saving the affected file.
2022-11-08 22:57:41 +01:00
greatroar
5dceadeb72
cmd: Don't read password from stdin for backup --stdin
2022-11-06 14:55:57 +01:00
Michael Eischer
1ccab95bc4
b2: Support file hiding instead of deleting them permanently
...
Automatically fall back to hiding files if not authorized to permanently
delete files. This allows using restic with an append-only application
key with B2. Thus, an attacker cannot directly delete backups with the
API key used by restic.
To use this feature create an application key without the deleteFiles
capability. It is recommended to restrict the key to just one bucket.
For example using the b2 command line tool:
b2 create-key --bucket <bucketName> <keyName> listBuckets,readFiles,writeFiles,listFiles
Suggested-by: Daniel Gröber <dxld@darkboxed.org>
2022-11-05 20:10:45 +01:00
Michael Eischer
24a2e5cab9
Merge pull request #4008 from MichaelEischer/tweak-lock-refresh-test
...
lock: Tweak timeouts for lock refresh test
2022-11-05 10:53:13 +01:00
Michael Eischer
403390479c
Merge pull request #3997 from greatroar/fuse-hash
...
fuse: Better check for whether snapshots changed
2022-11-05 10:52:11 +01:00
Michael Eischer
d29abc1a31
Merge pull request #4007 from MichaelEischer/hide-compression-level-for-v1-repo
...
Only print compression level starting from repository version 2
2022-11-05 10:33:25 +01:00
greatroar
c091e43b33
fuse: Better check for whether snapshots changed
...
We previously checked whether the set of snapshots might have changed
based only on their number, which fails when as many snapshots are
forgotten as are added. Check for the SHA-256 of their id's instead.
2022-11-05 09:32:45 +01:00
Michael Eischer
aaac63da8d
lock: Tweak timeouts for lock refresh test
...
For some reason the test fails from time to time. Increase the timeouts
to hopefully avoid this issue.
2022-11-04 22:48:18 +01:00
Michael Eischer
fd4d23460f
only print compression level starting from repository version 2
2022-11-04 22:40:07 +01:00
Alexander Neumann
8dd95b710e
Merge pull request #3992 from MichaelEischer/err-on-invalid-compression
...
Return error if RESTIC_COMPRESSION env variable is invalid
2022-11-04 19:41:34 +01:00
Alexander Neumann
783b8781a7
Merge pull request #4000 from restic/min-go-version
...
build: Correct checks for minimum Go version
2022-11-04 10:31:02 +01:00
Alexander Neumann
543649f2f2
Merge pull request #4001 from restic/docker-go-version
...
docker: Increase Go version to 1.19
2022-11-04 10:30:11 +01:00
Leo R. Lundgren
0a4cddb34d
docker: Increase Go version to 1.19
2022-11-03 22:59:59 +01:00
Leo R. Lundgren
333c2c6ed4
build: Correct checks for minimum Go version
2022-11-03 22:50:07 +01:00
rawtaz
92df039e5d
Merge pull request #3996 from MichaelEischer/fix-ui-progress
...
backup: fix stuck status bar
2022-11-02 21:48:16 +01:00
Michael Eischer
9354262b1b
backup: fix stuck status bar
...
The status bar got stuck once the first error was reported, the scanner
completed or some file was backed up. Either case sets a flag that the
scanner has started.
This flag is used to hide the progress bar until the flag is set. Due to
an inverted condition, the opposite happened and the status stopped
refreshing once the flag was set.
In addition, the scannerStarted flag was not set when the scanner just
reported progress information.
2022-11-02 21:31:13 +01:00
Michael Eischer
06141ce1f4
backup: print full snapshot id in JSON summary
2022-10-31 19:03:42 +01:00
Michael Eischer
59a90943bb
Merge pull request #3983 from greatroar/formatting
...
Centralize and fix formatting of bytes, percentages, durations
2022-10-31 18:52:24 +01:00
greatroar
5ab3e6276a
ui: Fix FormatBytes at exactly 1024 time a unit
...
1024 would be displayed as "1024 bytes" instead of "1.000 KiB", etc.
2022-10-31 18:39:28 +01:00
rawtaz
4f1fae9c98
Merge pull request #3982 from MichaelEischer/show-compression-mode
...
Show selected compression level when opening repository
2022-10-30 21:29:42 +01:00
Michael Eischer
8fe159cc5a
enable ysmlink tests for windows
2022-10-30 18:43:04 +01:00
Michael Eischer
3499c6354e
Merge pull request #3955 from MichaelEischer/async-futurefile-completion
...
Improve archiver performance for small files
2022-10-30 18:38:04 +01:00
Michael Eischer
144257f8bd
restore symlink timestamps on windows
2022-10-30 11:04:04 +01:00
Michael Eischer
c0f34af9db
backup: hide files from status which are read completely but not saved
...
As the FileSaver is asynchronously waiting for all blobs of a file to be
stored, the number of active files is higher than the number of files
from which restic is reading concurrently. Thus to not confuse users,
only display files in the status from which restic is currently reading.
2022-10-30 10:29:12 +01:00
Michael Eischer
a571fc4aa1
add changelog for faster backups with small files
2022-10-30 10:29:12 +01:00
Michael Eischer
b52a8ff05c
ui: Properly clear lines no longer used for status
...
Previously, the old status text remained until it was overwritten.
2022-10-30 10:29:12 +01:00
Michael Eischer
b4de902596
archiver: Asynchronously complete FutureFile
...
After reading and chunking all data in a file, the FutureFile still has
to wait until the FutureBlobs are completed. This was done synchronously
which results in blocking the file saver and prevents the next file from
being read.
By replacing the FutureBlob with a callback, it becomes possible to
complete the FutureFile asynchronously.
2022-10-30 10:29:11 +01:00
Michael Eischer
47e05080a9
Merge pull request #3990 from MichaelEischer/lock-refresh-test
...
lock: add test to check that refreshing works
2022-10-30 10:15:44 +01:00
Michael Eischer
c7ace314f6
Merge pull request #3989 from greatroar/eachbypack
...
More compact data structure for Index.EachByPack
2022-10-30 00:02:55 +02:00
greatroar
0e8893dae9
index: Compact data structure for Index.EachByPack
2022-10-29 23:09:17 +02:00
greatroar
137f0bc944
repository: Fix benchmarkSaveAndEncrypt
2022-10-29 23:09:17 +02:00
Michael Eischer
01f0db4e56
return error if RESTIC_COMPRESSION env variable is invalid
2022-10-29 22:03:39 +02:00
Michael Eischer
7c87fb941c
Merge pull request #3986 from greatroar/counter
...
ui/progress: Load both values in a single Lock/Unlock
2022-10-29 21:50:55 +02:00
Michael Eischer
3b0bb02a68
Merge pull request #3977 from greatroar/progress
...
ui/backup: Replace channels with a mutex
2022-10-29 21:33:04 +02:00
Michael Eischer
0d260cfd82
enable symlink test on windows
2022-10-29 21:26:34 +02:00
fgma
8e5eb1090c
issue2699: restore symlinks on windows when run as admin user
2022-10-29 21:19:33 +02:00
rawtaz
af3f7c866f
Merge pull request #3988 from FelixBurkhard/FelixBurkhard-patch-1
...
Clearify what Azure account name means
2022-10-29 13:32:44 +02:00
Michael Eischer
24267e9a9d
lock: add test to check that refreshing works
2022-10-29 11:26:00 +02:00
Michael Eischer
8e51e1e605
shorten 'repository opened' output
2022-10-29 11:22:00 +02:00
FelixBurkhard
575d26ec87
Clearify what Azure account name means
...
When reading it first it was not clear to me the 'account name' meant the name of the
Azure Storage Account and not the Azure account itself.
2022-10-29 00:27:43 +02:00
greatroar
2dafda9164
ui/progress: Load both values in a single Lock/Unlock
...
We always need both values, except in a test, so we don't need to lock
twice and risk scheduling in between.
Also, removed the resetting in Done. This copied a mutex, which isn't
allowed. Static analyzers tend to trip over that.
2022-10-25 07:55:24 +02:00
Michael Eischer
f8910bc4ff
Merge pull request #3985 from saltsa/fix_lock_refresh
...
Fix bug in lock refresh monitoring
2022-10-24 22:59:18 +02:00
Joonas Aunola
b06427c9f6
fix Unix to UnixNano
2022-10-23 23:40:21 +03:00
greatroar
006380199e
cmd, ui: Deduplicate formatting utilities
2022-10-23 13:40:07 +02:00
greatroar
04216eb9aa
ui/backup: Replace channels with a mutex
...
The channel-based algorithm had grown quite complicated. This is easier
to reason about and likely to be more performant with very many
CompleteBlob calls.
2022-10-23 13:28:41 +02:00
Michael Eischer
4fea3a413d
show selected compression level when opening repository
2022-10-22 20:18:46 +02:00
Michael Eischer
ba58ccbe07
prune: add remark about non-deterministic blob selection
2022-10-22 19:46:10 +02:00
Michael Eischer
05651d6d4f
prune: Correctly count used/duplicate blobs for partially compressed repos
...
Counting the first occurrence of a duplicate blob as used and counting
all other as duplicates, independent of which instance of the blob is
kept, is only accurate if all copies of the blob have the same size. This
is no longer the case for a repository containing both compressed and
uncompressed blobs.
Thus for duplicated blobs first count all instances as duplicates and
then subtract the actually used instance later on.
2022-10-22 19:24:36 +02:00
Michael Eischer
b57d42905c
Merge pull request #3899 from MichaelEischer/less-prune-mem
...
Optimize prune memory usage
2022-10-22 18:56:02 +02:00
Michael Eischer
d966c52707
prune: allow gc of set of repacked blobs before index rebuild
2022-10-22 18:45:12 +02:00
Michael Eischer
1e2794fa55
add prune memory optimization changelog
2022-10-22 18:45:12 +02:00
Michael Eischer
68c9cb9c6a
prune: Shrink keepBlobs set if possible
...
As long as only a small fraction of the data in a repository is
rewritten, the keepBlobs set will be rather small after cleaning it up.
As golang maps do not shrink their memory usage, just copy the contents
over to a new map. However, only copy the map if the cleanup removed at
least half the entries.
2022-10-22 18:45:12 +02:00
Michael Eischer
c4fc5c97f9
prune: Use a single CountedBlobSet to track blobs
...
The set covers necessary, existing and duplicate blobs. This removes the
duplicate sets used to track whether all necessary blobs also exist.
This reduces the memory usage of prune by about 20-30%.
2022-10-22 18:45:12 +02:00
Michael Eischer
b21241ec1c
restic: Add CountedBlobSet type
...
This allows maintaining a usage counter for each blob.
2022-10-22 18:45:12 +02:00
Michael Eischer
ee6688a9f6
Merge pull request #3915 from plumbeo/compression-stats
...
restic stats: print uncompressed size in mode raw-data
2022-10-21 22:10:29 +02:00
Michael Eischer
27634a1a68
Merge pull request #3978 from MichaelEischer/fix-negative-pattern-example
...
Remove misleading wildcard from negative exclude pattern example
2022-10-21 22:04:30 +02:00
Michael Eischer
aa77702e49
Merge pull request #3971 from MichaelEischer/parallel-list
...
Unify ForAllIndex/Snapshot/Lock functions
2022-10-21 21:58:33 +02:00
Michael Eischer
6877aaa8aa
Merge pull request #3967 from MichaelEischer/archiver-extract-exclude-options
...
backup: extract exclude pattern options
2022-10-21 21:50:00 +02:00
Michael Eischer
2e9ee8577a
Merge pull request #3970 from MichaelEischer/split-retry-backend
...
Split backend package into smaller parts
2022-10-21 21:49:46 +02:00
Michael Eischer
59d46bb3f5
backup: extract exclude pattern options
...
This is a preparation to make the exclude options usable for the
upcoming `rewrite` command.
2022-10-21 21:40:59 +02:00
Michael Eischer
5c7a9a739a
backend: Split RetryBackend into own package
...
The RetryBackend tests depend on the mock backend. When the Backend
interface is eventually split from the restic package, this will lead to
a dependency cycle between backend and backend/mock. Thus split the
RetryBackend into a separate package to avoid this problem.
2022-10-21 21:38:17 +02:00
Michael Eischer
32603d49c4
backend: remove unused ErrorBackend
2022-10-21 21:36:05 +02:00
Michael Eischer
8c18c65b3b
backend: remove unused Paths variable
2022-10-21 21:36:05 +02:00
Michael Eischer
4ccd5e806b
backend: split layout code into own subpackage
2022-10-21 21:36:05 +02:00
Michael Eischer
b361284f28
Merge pull request #3979 from MichaelEischer/backup-less-time-now
...
backup: reduce calls to time.Now
2022-10-21 21:33:34 +02:00
Michael Eischer
738b2a0445
parallelize more List usages
2022-10-21 21:26:45 +02:00
Michael Eischer
ae45f3b04f
restic: Unify code to load Index/Lock/Snapshot
2022-10-21 21:25:11 +02:00
Michael Eischer
8e2695be0b
Merge pull request #3973 from MichaelEischer/speedup-integration-tests
...
speed-up integration tests by reducing the RetryBackend timeout
2022-10-21 21:17:35 +02:00
Michael Eischer
35d968bcde
Merge pull request #3969 from MichaelEischer/key-by-id
...
Port restic.Find to return IDs and identify keys by restic.ID
2022-10-21 21:15:40 +02:00
Michael Eischer
4133fee6f9
Merge pull request #3972 from MichaelEischer/fix-flaky-lock-cancel-test
...
lock: fix flaky TestLockFailedRefresh
2022-10-21 21:12:34 +02:00
Michael Eischer
c8c8391b21
Merge pull request #3974 from greatroar/cleanup
...
More cleanups and a micro-optimization
2022-10-21 21:11:37 +02:00
Michael Eischer
ee7c28f5e6
backup: reduce calls to time.Now
...
Archiver.Save queries the current time multiple times. This commit
removes one of these calls as they showed up while profiling a backup of
a nearly unchanged dataset containing 3 million files.
2022-10-21 20:55:01 +02:00
Michael Eischer
3e60d38a23
Remove misleading wildcard from negative exclude pattern example
...
There is no need to use a special wildcard `**` to demonstrate negative
patterns. Actually, it is both slower than the simpler variant and seems
to confuse users.
2022-10-21 20:48:45 +02:00
greatroar
9adae5521d
cache: Call interface method once
2022-10-21 14:32:46 +02:00
greatroar
201e5c7e74
backup: Clean up progress reporting code
2022-10-21 13:48:30 +02:00
plumbeo
a6f83e0011
Add changelog
2022-10-17 15:38:42 +02:00
plumbeo
bc945d0bf0
restic stats: add more compression statistics
...
Calculate and display compression ratio, space saving and progress
2022-10-17 15:38:38 +02:00
greatroar
b513597546
internal/restic: Make FileType a uint8 instead of a string
...
The string form was presumably useful before the introduction of
layouts, but right now it just makes call sequences and garbage
collection more expensive (the latter because every string contains
a pointer to be scanned).
2022-10-16 10:59:01 +02:00
greatroar
22147e1e02
all: Minor cleanups
...
if x { return true } return false => return x
fmt.Sprintf("%v", x) => fmt.Sprint(x) or x.String()
The fmt.Sprintf idiom is still used in the SecretString tests, where it
serves security hardening.
2022-10-16 10:50:39 +02:00
greatroar
d03460010f
internal/restic: Fix ID.UnmarshalJSON, ParseID
...
ID.UnmarshalJSON accepted non-JSON input with ' as the string delimiter.
Also, the error message for non-hex input was less informative than it
could be and it performed too many checks.
Changed ParseID to keep the error messages consistent.
2022-10-16 10:39:52 +02:00
Michael Eischer
aa39bf3cf6
backend/test: remove duplicate test
...
The test is identical to the tests for the mem backend.
2022-10-15 23:15:07 +02:00
Michael Eischer
28e1c4574b
mem: use cheaper hash for backend
2022-10-15 23:14:33 +02:00
Michael Eischer
c3400d3c55
backend: speedup RetryBackend tests
2022-10-15 23:13:44 +02:00
Michael Eischer
99547518cd
lock: fix flaky TestLockFailedRefresh
...
The comparison of the current time and the last lock refresh were using
seconds represented as integers. As the test only waits for up to one
second, the associated number truncation can cause the test to take
longer than once second and thus to fail.
Switch to nanoseconds to avoid this problem. This also slightly speeds
up the test.
2022-10-15 22:36:32 +02:00
Michael Eischer
e10420553b
speed-up integration tests by reducing the RetryBackend timeout
...
On my machine this decreases the runtime for `./cmd/restic` from 9.5s to
6.5s.
2022-10-15 22:29:58 +02:00
Michael Eischer
367f35db27
Merge pull request #3968 from MichaelEischer/cleanup-complete-blob
...
backup: Remove unused filename parameter from CompleteBlob callback
2022-10-15 16:11:16 +02:00
Michael Eischer
8d62a7adb4
identify keys by ID and not name
2022-10-15 16:07:43 +02:00
Michael Eischer
02634dce7a
restic: change Find to return ids
...
That way consumers no longer have to manually convert the returned name
to an id.
2022-10-15 16:06:54 +02:00
Michael Eischer
964977677f
backup: Remove unused filename parameter from CompleteBlob callback
2022-10-15 15:21:17 +02:00
Michael Eischer
258b487d8f
Merge pull request #3951 from MichaelEischer/rework-snapshot-filter
...
Rework snapshot filtering
2022-10-15 14:47:47 +02:00
Michael Eischer
de9bc031df
add changelog for ls handling of missing snapshots
2022-10-15 13:34:50 +02:00
Michael Eischer
246d3032ae
restic: Don't list snapshots if FindSnapshot gets full id
2022-10-15 13:34:34 +02:00
Michael Eischer
d8c00b9726
add comment
2022-10-15 13:34:21 +02:00
Michael Eischer
a3113c6097
restic: Change FindSnapshot functions to return the snapshot
2022-10-15 13:34:04 +02:00
Michael Eischer
b50f48594d
restic: cleanup arguments of findLatestSnapshot
2022-10-15 13:33:48 +02:00
Michael Eischer
61e827ae4f
restic: hide findLatestSnapshot
2022-10-15 13:33:32 +02:00
Michael Eischer
fcad5e6f5d
backup: use unified FindFilteredSnapshot
2022-10-15 13:33:29 +02:00
Michael Eischer
0aa73bbd39
ls: proper error handling for non-existent snapshot
...
Use restic.FindFilteredSnapshot to resolve the snapshot ID. This ensures
consistent behavior for all commands using initSingleSnapshotFilterOptions.
2022-10-15 13:32:00 +02:00
Michael Eischer
a81f0432e9
restic: Add unified method to resolve a single snapshot
2022-10-15 13:31:45 +02:00
Michael Eischer
95a1bb4261
restic: Rework error handling of FindFilteredSnapshots and handle snapshotIDs
...
FindFilteredSnapshots no longer prints errors during snapshot loading on
stderr, but instead passes the error to the callback to allow the caller
to decide on what to do.
In addition, it moves the logic to handle an explicit snapshot list from
the main package to restic.
2022-10-15 13:31:26 +02:00
Michael Eischer
cff22a5f01
dump: use correct help text for filter options
2022-10-15 13:31:10 +02:00
Michael Eischer
7a6dcb4831
Merge pull request #3966 from MichaelEischer/cleanup-walker-test
...
walker: Convert tests to use TreeJSONBuilder
2022-10-15 11:25:11 +02:00
Michael Eischer
7cf042118f
walker: Convert tests to use TreeJSONBuilder
...
The old code marshalled the tree blobs different than other places in
restic. The hashed tree blob did not contain a final newline character.
2022-10-15 11:04:13 +02:00
Michael Eischer
cea7191995
Merge pull request #3959 from MichaelEischer/buffered-backup-progress
...
backup: Use buffered channels to collect backup status
2022-10-15 10:57:19 +02:00
Michael Eischer
ba688aad20
Merge pull request #3961 from greatroar/cleanup
...
Misc. cleanup
2022-10-14 21:49:35 +02:00
Michael Eischer
9c290a8093
Merge pull request #3960 from greatroar/errors
...
errors: Drop WithMessage
2022-10-14 21:41:28 +02:00
greatroar
0e155fd9a6
internal/restic: Fix UID/GID parsing
...
The helper function uidGidInt used strconv.ParseInt instead of
ParseUint, so it silently ignored some invalid user/group IDs.
Also, improve the error message. "Invalid UID" is more informative than
having "ParseInt" twice (*strconv.NumError displays the function name).
Finally, the user.User struct can be passed by pointer to get reduce
code size.
2022-10-14 18:21:00 +02:00
greatroar
e0b743c64d
internal/restic: Remove unused ID.EqualString
2022-10-14 18:20:11 +02:00
greatroar
6922360179
ui/backup: Remove unused ProgressReporter type, Progress field
2022-10-14 14:36:19 +02:00
greatroar
d4aadfa389
all: Drop ctxhttp
...
This package is no longer needed, since we can use the stdlib's
http.NewRequestWithContext.
backend/rclone already did, but it needed a different error check due to
a difference between net/http and ctxhttp.
Also, store the http.Client by value in the REST backend (changed to a
pointer when ctxhttp was introduced) and use errors.WithStack instead
of errors.Wrap where the message was no longer accurate. Errors from
http.NewRequestWithContext will start with "net/http" or "net/url", so
they're easy to identify.
2022-10-14 14:33:49 +02:00
greatroar
16849d5361
internal/archiver: Missing argument to errors.Errorf
2022-10-14 14:18:52 +02:00
greatroar
09c14f33c8
internal/checker: Pass Error.Error pointer receiver
2022-10-14 14:13:32 +02:00
greatroar
feb790f497
internal/restic: Use errors.New when no formatting is needed
2022-10-14 14:07:20 +02:00
greatroar
ba44666704
errors: Drop WithMessage
2022-10-14 14:06:47 +02:00
Michael Eischer
1a6160d152
Merge pull request #3880 from MichaelEischer/archiver-savedir-cleanup
...
archiver: Improve handling of "file xxx already present" error
2022-10-08 21:48:14 +02:00
Michael Eischer
21b1d7a880
Merge pull request #3948 from MichaelEischer/split-index
...
repository: split index into a separate package
2022-10-08 21:41:57 +02:00
Michael Eischer
5278ab51c8
archiver: Check that duplicates are only ignored if identical
2022-10-08 21:38:36 +02:00
Michael Eischer
403b01b788
backup: Only return a warning for duplicate directory entries
...
The backup command failed if a directory contains duplicate entries.
Downgrade the severity of this problem from fatal error to a warning.
This allows users to still create a backup.
2022-10-08 21:38:21 +02:00
Michael Eischer
d7d7b4ab27
archiver: refactor TreeSaverTest
2022-10-08 21:29:32 +02:00
Michael Eischer
8e38c43c27
archiver: let FutureNode.Take return an error if no data is available
...
This ensures that we cannot accidentally store an invalid node.
2022-10-08 21:28:39 +02:00
Michael Eischer
2b88cd6eab
archiver: Restructure SaveTree to work like SaveDir
...
SaveTree did not use the TreeSaver but rather managed the tree
collection and upload itself. This prevents using the parallelism
offered by the TreeSaver and duplicates all related code. Using the
TreeSaver can provide some speed-ups as all steps within the backup tree
now rely on FutureNodes. This can be especially relevant for backups
with large amounts of explicitly specified files.
The main difference between SaveTree and SaveDir is, that only the
former can save tree blobs in which nodes have a different name than the
actual file on disk. This is the result of resolving name conflicts
between multiple files with the same name. The filename that must be
used within the snapshot is now passed directly to
restic.NodeFromFileInfo. This ensures that a FutureNode already contains
the correct filename.
2022-10-08 21:28:39 +02:00
Michael Eischer
2e3f1c08c5
repository: split index into a separate package
2022-10-08 21:15:34 +02:00
Michael Eischer
5760ba6989
Merge pull request #3949 from MichaelEischer/simplify-mixedpacks
...
repository: remove IsMixedPack and add replacement for checker
2022-10-08 21:14:14 +02:00
Michael Eischer
5ee25e669a
Merge pull request #3940 from MichaelEischer/better-rclone-error
...
Better error message if connection to rclone fails
2022-10-08 21:14:00 +02:00
Michael Eischer
5600f11696
rclone: Fix stderr handling if command exits unexpectedly
...
According to the documentation of exec.Cmd Wait() must not be called
before completing all reads from the pipe returned by StdErrPipe(). Thus
return a context that is canceled once rclone has exited and use that as
a precondition to calling Wait(). This should ensure that all errors
printed to stderr have been copied first.
2022-10-08 20:16:06 +02:00
Michael Eischer
b8acad4da0
rclone: return rclone error instead of canceled context
...
When rclone fails during the connection setup this currently often
results in a context canceled error. Replace this error with the exit
code from rclone.
2022-10-08 20:15:24 +02:00
Michael Eischer
d3ebec8f21
backup: Use buffered channels to collect backup status
...
When backing up many small files, the unbuffered channels frequently
cause the FileSaver to block when reporting progress information. Thus,
add buffers to these channels to avoid unnecessary scheduling.
As the status information is purely informational, it doesn't matter
that the status reporting shutdown is somewhat racy and could miss a few
final updates.
2022-10-08 18:20:41 +02:00
Michael Eischer
f9d4e0c2af
Merge pull request #3958 from greatroar/errors
...
errors: Drop Cause in favor of Go 1.13 error handling
2022-10-08 18:06:35 +02:00
Michael Eischer
119e6aee01
Merge pull request #3957 from greatroar/typo
...
cmd: Typo in --read-concurrency description
2022-10-08 14:41:35 +02:00
greatroar
07e5c38361
errors: Drop Cause in favor of Go 1.13 error handling
...
The only use cases in the code were in errors.IsFatal, backend/b2,
which needs a workaround, and backend.ParseLayout. The last of these
requires all backends to implement error unwrapping in IsNotExist.
All backends except gs already did that.
2022-10-08 13:08:08 +02:00
greatroar
4eae4d3e1a
cmd: Typo in --read-concurrency description
2022-10-08 11:27:39 +02:00
Michael Eischer
83cb58b4f3
Merge pull request #3956 from MichaelEischer/fix-lock-refresh
...
lock: Use the correct duration to check for expired locks
2022-10-07 22:58:10 +02:00
Michael Eischer
7c5d63a794
lock: Use the correct duration to check for expired locks
2022-10-07 22:39:53 +02:00
Michael Eischer
8b7c952f17
Merge pull request #3953 from keachi/typo
...
Fix typo
2022-10-07 22:18:32 +02:00
Michael Eischer
e43d2d45f7
Merge pull request #3952 from hoelzro/master
...
Update copy documentation to use --from-repo option
2022-10-07 22:18:06 +02:00
Rob Hoelz
03e9a26018
Update copy documentation to use --from-repo option
...
Removing the last references to the deprecated --repo2 option
2022-10-07 22:00:12 +02:00
tr
43cc01d63e
doc: Fix typo
2022-10-05 21:03:14 +02:00
Michael Eischer
7112a132c3
Merge pull request #3950 from MichaelEischer/misc-cleanups
...
Cleanups for cmd_debug/repository and remove dead code from restic package
2022-10-03 12:46:32 +02:00
Michael Eischer
4bb5240720
repository: remove unused PrefixLength
2022-10-03 12:15:53 +02:00
Michael Eischer
999fe29976
repository: hide prepareCache
2022-10-03 12:15:53 +02:00
Michael Eischer
9197c63007
debug: use repository.ListPack wrapper
2022-10-03 12:09:08 +02:00
Michael Eischer
ddcf549eba
repository: remove IsMixedPack and add replacement for checker
...
Repositories with mixed packs are probably quite rare by now. When
loading data blobs from a mixed pack file, this will no longer trigger
caching that file. However, usually tree blobs are accessed first such
that this shouldn't make much of a difference.
The checker gets a simpler replacement.
2022-10-03 12:03:59 +02:00
Michael Eischer
a61fbd287a
Merge pull request #3569 from MichaelEischer/strict-locking
...
Strict repository lock handling
2022-10-03 00:44:44 +02:00
Michael Eischer
6d2d297215
pass global context through cobra
2022-10-03 00:19:46 +02:00
Michael Eischer
49126796d0
lock: fix timer expiry monitoring during standby
...
Monotonic timers are paused during standby. Thus these timers won't fire
after waking up. Fall back to periodic polling to detect too large clock
jumps. See https://github.com/golang/go/issues/35012 for a discussion of
go timers during standby.
2022-10-03 00:19:46 +02:00
Michael Eischer
401e432e9d
lock: Do not ignore invalid lock files
...
While searching for lock file from concurrently running restic
instances, restic ignored unreadable lock files. These can either be
in fact invalid or just be temporarily unreadable. As it is not really
possible to differentiate between both cases, just err on the side of
caution and consider the repository as already locked.
The code retries searching for other locks up to three times to smooth
out temporarily unreadable lock files.
2022-10-03 00:19:46 +02:00
Michael Eischer
aeed420e1a
add changelog
2022-10-03 00:19:46 +02:00
Michael Eischer
9959190e39
lock: Add integration test
...
The tests check that the wrapped context is properly canceled whenever
the repository is unlock or when the lock refresh fails.
2022-10-03 00:19:46 +02:00
Michael Eischer
c3538b063a
lock: Use repository interface instead of struct
2022-10-03 00:19:46 +02:00
Michael Eischer
d92957dd78
lock: Implement strict lock expiry monitoring
...
Restic continued e.g. a backup task even when it failed to renew the
lock or failed to do so in time. For example if a backup client enters
standby during the backup this can allow other operations like `prune`
to run in the meantime (after calling `unlock`). After leaving standby
the backup client will continue its backup and upload indexes which
refer pack files that were removed in the meantime.
This commit introduces a goroutine explicitly monitoring for locks that
are not refreshed in time. To simplify the implementation there's now a
separate goroutine to refresh the lock and monitor for timeouts for each
lock. The monitoring goroutine would now cause the backup to fail as the
client has lost it's lock in the meantime.
The lock refresh goroutines are bound to the context used to lock the
repository initially. The context returned by `lockRepo` is also
cancelled when any of the goroutines exits. This ensures that the
context is cancelled whenever for any reason the lock is no longer
refreshed.
2022-10-03 00:19:46 +02:00
Michael Eischer
928914f821
Prepare for context bound to lock lifetime
2022-10-03 00:19:46 +02:00
Michael Eischer
985722b102
Remove ctx from globalOptions
...
Previously the global context was either accessed via gopts.ctx,
stored in a local variable and then used within that function or
sometimes both. This makes it very hard to follow which ctx or a wrapped
version of it reaches which method.
Thus just drop the context from the globalOptions struct and pass it
explicitly to every command line handler method.
2022-10-03 00:19:46 +02:00
Michael Eischer
ab819b2344
key: Cleanup method signatures
2022-10-03 00:19:46 +02:00
Michael Eischer
d0668b695d
Remove unnecessary context.WithCancel calls
...
The gopts.ctx is cancelled when the main() method of restic exits.
2022-10-03 00:19:46 +02:00
Michael Eischer
7ce4cb7908
Merge pull request #3947 from MichaelEischer/fix-cache-verify-test
...
cache: Fix file descriptor leak in TestBackendRemoveBroken
2022-10-03 00:19:26 +02:00
Michael Eischer
430ab32941
cache: Fix file descriptor leak in TestBackendRemoveBroken
2022-10-03 00:06:44 +02:00
Michael Eischer
e99ad39b34
Merge pull request #2750 from metalsp0rk/min-packsize
...
Add `backup --file-read-concurrency` flag
2022-10-02 23:11:47 +02:00
Michael Eischer
2e606ca70b
backup: rework read concurrency
2022-10-02 22:55:14 +02:00
Kyle Brennan
4a501d7118
backup: add option for file read concurrency
2022-10-02 22:51:45 +02:00
Michael Eischer
9ec7eee803
Merge pull request #3521 from MichaelEischer/redownload-broken-files
...
Redownload files with wrong hash
2022-10-02 22:50:03 +02:00
Michael Eischer
b25d0773b6
Merge pull request #3944 from MichaelEischer/fix-linter-errors
...
CI: ignore warning about missing package comment
2022-09-27 21:41:55 +02:00
Michael Eischer
5265550ff3
CI: ignore warning about missing package comment
2022-09-27 21:31:37 +02:00
Michael Eischer
e89fc2a29d
Merge pull request #3943 from MichaelEischer/find-match-only-valid-ids
...
ignore filenames which are not IDs when expanding a prefix
2022-09-27 20:56:48 +02:00
Michael Eischer
67e4620cd6
Merge pull request #3938 from restic/errdot
...
rclone/sftp: Improve handling of ErrDot errors
2022-09-27 20:33:42 +02:00
Michael Eischer
5d3c5b9e50
restic: ignore filenames which are not IDs when expanding a prefix
...
Some backends generate additional files for each existing file, e.g.
1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef.sha256
For some commands this leads to an "multiple IDs with prefix" error when
trying to reference a snapshot.
2022-09-27 20:30:40 +02:00
Leo R. Lundgren
ebe9f2c969
rclone/sftp: Improve handling of ErrDot errors
...
Restic now yields a more informative error message when exec.ErrDot occurs.
2022-09-25 16:19:03 +02:00
Michael Eischer
d114e483c4
Add changelog for corrupt data downloads
2022-09-25 11:55:09 +02:00
Michael Eischer
34c1a83340
cache: Drop cache entry if it cannot be processed
...
Failing to process data requested from the cache usually indicates a
problem with the returned data. Assume that the cache entry is somehow
damaged and retry downloading it once.
2022-09-25 11:55:09 +02:00
Michael Eischer
aa3b1925b4
cache: Simplify loadFromCacheOrDelegate
2022-09-25 11:35:35 +02:00
Michael Eischer
5c6b6edefe
retry index, lock and snapshot loading on hash mismatch
2022-09-25 11:35:35 +02:00
Michael Eischer
822422ef03
retry key loading on hash mismatch
2022-09-25 11:35:35 +02:00
Michael Eischer
d6575f53ca
Merge pull request #3942 from MichaelEischer/split-cross-compile-test
...
Split cross compile test
2022-09-24 22:27:08 +02:00
Michael Eischer
78d2312ee9
Merge pull request #3854 from MichaelEischer/sparsefiles
...
restore: Add support for sparse files
2022-09-24 22:04:02 +02:00
Michael Eischer
46b30b9826
split cross compilation into three parts
...
The cross compilation tasks are currently the slowest part of the CI
runs. Splitting it into three parts should reduce its time to roughly
that of the windows CI run.
2022-09-24 22:00:25 +02:00
Michael Eischer
bd191ec60b
update golang-ci to version 1.49
2022-09-24 22:00:08 +02:00
Michael Eischer
519059cca4
update ci actions
2022-09-24 21:59:36 +02:00
Michael Eischer
19afad8a09
restore: support sparse restores also on windows
2022-09-24 21:39:39 +02:00
Michael Eischer
0f89f443c7
update sparse restore changelog
2022-09-24 21:39:39 +02:00
Michael Eischer
c147422ba5
repository: special case SaveBlob for all zero chunks
...
Sparse files contain large regions containing only zero bytes. Checking
that a blob only contains zeros is possible with over 100GB/s for modern
x86 CPUs. Calculating sha256 hashes is only possible with 500MB/s (or
2GB/s using hardware acceleration). Thus we can speed up the hash
calculation for all zero blobs (which always have length
chunker.MinSize) by checking for zero bytes and then using the
precomputed hash.
The all zeros check is only performed for blobs with the minimal chunk
size, and thus should add no overhead most of the time. For chunks which
are not all zero but have the minimal chunks size, the overhead will be
below 2% based on the above performance numbers.
This allows reading sparse sections of files as fast as the kernel can
return data to us. On my system using BTRFS this resulted in about
4GB/s.
2022-09-24 21:39:39 +02:00
Michael Eischer
34fe1362da
restorer: move zeroPrefixLen to restic package
2022-09-24 21:39:39 +02:00
Michael Eischer
a5ebd5de4b
restorer: Fix race condition in partialFile.WriteAt
...
The restorer can issue multiple calls to WriteAt in parallel. This can
result in unexpected orderings of the Truncate and WriteAt calls and
sometimes too short restored files.
2022-09-24 21:39:39 +02:00
Michael Eischer
5b6a77058a
Enable sparseness only conditionally
...
We can either preallocate storage for a file or sparsify it. This
detects a pack file as sparse if it contains an all zero block or
consists of only one block. As the file sparsification is just an
approximation, hide it behind a `--sparse` parameter.
2022-09-24 21:20:00 +02:00
greatroar
3047bf611c
Changelog entry for sparse file restoring
2022-09-24 21:18:48 +02:00
greatroar
5d4568d393
Write sparse files in restorer
...
This writes files by using (*os.File).Truncate, which resolves to the
truncate system call on Unix.
Compared to the naive loop,
for _, b := range p {
if b != 0 {
return false
}
}
the optimized allZero is about 10× faster:
name old time/op new time/op delta
AllZero-8 1.09ms ± 1% 0.09ms ± 1% -92.10% (p=0.000 n=10+10)
name old speed new speed delta
AllZero-8 3.84GB/s ± 1% 48.59GB/s ± 1% +1166.51% (p=0.000 n=10+10)
2022-09-24 21:18:48 +02:00
Michael Eischer
eb83402d39
Merge pull request #3935 from miles170/master
...
Only display the message if there were locks to be removed
2022-09-24 20:53:13 +02:00
Michael Eischer
ef58ddd7b1
Merge pull request #3923 from MichaelEischer/fix-flaky-cache-test
...
cache: fix flaky TestFileSaveConcurrent on windows
2022-09-24 20:52:55 +02:00
Michael Eischer
7fc178aaf4
internal/cache: extend description of cache sharing test failure
2022-09-24 13:07:01 +02:00
Miles Liu
1acbda18f8
Only display the message if there were locks to be removed
...
`restic unlock` now only shows `successfully removed locks` if there were locks to be removed.
In addition, it also reports the number of the removed lock files.
2022-09-24 19:02:24 +08:00
Michael Eischer
da1a359c8b
Merge pull request #3927 from MichaelEischer/faster-index-each
...
Speed up MasterIndex.Each
2022-09-24 12:35:23 +02:00
Michael Eischer
041a51512a
Merge pull request #3780 from jkmw/fix/2578
...
Remove existing path before restoring a symlink
2022-09-24 12:34:42 +02:00
Michael Eischer
1ebd57247a
repository: optimize MasterIndex.Each
...
Sending data through a channel at very high frequency is extremely
inefficient. Thus use simple callbacks instead of channels.
> name old time/op new time/op delta
> MasterIndexEach-16 6.68s ±24% 0.96s ± 2% -85.64% (p=0.008 n=5+5)
2022-09-24 12:21:59 +02:00
Michael Eischer
825b95e313
repository: add benchmark for MasterIndex.Each
2022-09-24 12:21:59 +02:00
greatroar
1220fe9650
internal/cache: Concurrent use of cache not working on Windows
2022-09-17 19:49:44 +02:00
Jerome Küttner
ef618bdd3f
use os.Remove if path already exists on symlink restore
2022-09-14 08:14:31 +02:00
Michael Eischer
b48766d7b8
Merge pull request #3928 from restic/rawtaz-doc-b2-s3
...
doc: Clarify S3 recommendation for B2 slightly
2022-09-13 20:59:50 +02:00
rawtaz
20f1913ef7
doc: Clarify S3 recommendation for B2 slightly
...
This gives slightly more background to the recommendation for using restic's S3 backend with Backblaze B2.
2022-09-12 17:48:59 +02:00
rawtaz
d79e61ce5d
Merge pull request #3925 from hgraeber/add-powershell-completion
...
Add powershell completion
2022-09-11 01:04:57 +02:00
Herbert Graeber
988b386e8b
Add powershell completion
...
- Add code for powersehll complition available in cobra
- Add documentation for powershell completion
- Add changelog for pr3925
2022-09-11 00:44:12 +02:00
rawtaz
14d09a6081
Merge pull request #3912 from MichaelEischer/cleanup-snapshot-filter-options
...
Clean up snapshot filter options
2022-09-11 00:18:42 +02:00
Michael Eischer
381da0443a
tweak snapshot filter descriptions
2022-09-10 23:50:20 +02:00
Michael Eischer
8b9778d537
Merge pull request #3900 from MichaelEischer/b2-init-timeout
...
Add timeout for the initial connection to B2
2022-09-10 23:28:59 +02:00
Michael Eischer
17c27400f8
Merge pull request #3921 from MichaelEischer/filter-cleanup-error-handling
...
filter: deduplicate error handling for pattern validation
2022-09-10 23:24:50 +02:00
Michael Eischer
f76643bd2e
Merge pull request #3894 from MichaelEischer/filter-mount-exit-code
...
Mount should return exit code 0 after pressing Ctrl-C
2022-09-10 23:22:01 +02:00
Michael Eischer
be9ccc186e
Merge pull request #3875 from MichaelEischer/fix-fuse-context-cancel
...
mount: Fix input/output errors for canceled syscalls
2022-09-10 23:20:29 +02:00
Michael Eischer
2363e5c083
Merge pull request #3913 from MichaelEischer/better-migrate-error-message
...
migrate: Report why an migration cannot be applied
2022-09-09 23:37:25 +02:00
Michael Eischer
8e0ca80547
filter: deduplicate error handling for pattern validation
2022-09-09 23:12:41 +02:00
plumbeo
d66e755ac7
Change uncompressed size calculation to account for the encryption overhead
2022-09-08 10:15:19 +02:00
plumbeo
837b816358
restic stats: print uncompressed size in mode raw-data
2022-09-05 17:38:32 +02:00
Michael Eischer
d6309961c5
deduplicate the snapshot filter cli option setup
2022-09-04 10:27:33 +02:00
Michael Eischer
8b4dd70013
migrate: Report why an migration cannot be applied
...
Just returning that `Migration upgrade cannot be applied: check failed`
is not too useful when running `migrate upgrade_repo_v2`.
2022-09-03 11:49:31 +02:00
Michael Eischer
7689d6c679
normalize help text for host, tag and path options
2022-09-03 00:06:38 +02:00
Michael Eischer
6c69f08a7b
Merge pull request #3905 from DRON-666/haspaths-linear
...
Reduce quadratic time complexity of `Snapshot.HasPaths`
2022-08-30 20:35:56 +02:00
Michael Eischer
3e70bac56e
Merge pull request #3898 from MichaelEischer/fix-copy-hang
...
don't hang when `copy` uses a single connection
2022-08-30 20:23:39 +02:00
DRON-666
2a630c51c1
Add changelog
2022-08-30 20:22:07 +02:00
DRON-666
d0f1060df7
Fix quadratic time complexity of Snapshot.HasPaths
2022-08-30 04:38:17 +03:00
Michael Eischer
f481ad64c8
Merge pull request #3904 from lbausch/add-newline
...
Add newline to keep prompt intact
2022-08-29 21:43:18 +02:00
Lorenz Bausch
7ddd803e46
Add newline to keep prompt intact
2022-08-29 17:37:49 +02:00
Michael Eischer
e5b2c4d571
b2: sniff the error that caused init retry loops
2022-08-28 17:46:03 +02:00
Michael Eischer
dc2db2de5e
b2: cancel connection setup after a minute
...
If the connection to B2 fails, the library enters an endless loop.
2022-08-28 14:56:17 +02:00
Michael Eischer
7682149c9d
repository: cleanup copy connection count check
2022-08-28 11:40:56 +02:00
Michael Eischer
b03277ead5
repository: don't hang when copying using a single connection
2022-08-28 11:40:31 +02:00
Michael Eischer
1b233c4e2e
Merge pull request #2661 from creativeprojects/issue-1734
...
"Everything is ok" message after retrying
2022-08-28 11:04:59 +02:00
Fred
4042db5169
Add changelog
2022-08-27 22:36:19 +02:00
Fred
be6baaec12
Add success callback to the backend
2022-08-27 22:27:15 +02:00
Fred
baf58fbaa8
Add unit tests
2022-08-27 22:21:06 +02:00
Fred
d629333efe
Add function to notify of success after retrying
2022-08-27 22:21:06 +02:00
Alexander Neumann
c169e37139
Merge pull request #3895 from MichaelEischer/refactor-cat-key
...
cat: Simplify implementation of 'cat key'
2022-08-27 18:40:46 +02:00
Michael Eischer
1b4af0c6e5
cat: Simplify implementation of 'cat key'
2022-08-26 23:21:51 +02:00
Michael Eischer
3174641ca4
add changelog for mount exit code filtering
2022-08-26 23:17:04 +02:00
Michael Eischer
5478ab22c5
mount: return exit code 0 after receiving a SIGINT
2022-08-26 23:07:07 +02:00
Michael Eischer
d768c1c3e4
Allow cleanup handlers to filter the exit code
2022-08-26 23:04:59 +02:00
Michael Eischer
908f7441fe
Merge pull request #3885 from MichaelEischer/delete-fixes
...
Improve reliability of upload retries and B2 file deletions
2022-08-26 22:30:50 +02:00
Michael Eischer
4c90d91d4d
backend: Test that failed uploads are not removed for backends with atomic replace
2022-08-26 21:20:52 +02:00
Michael Eischer
694dfa026a
add changelog for reliable B2 deletes
2022-08-26 21:20:46 +02:00
MichaelEischer
582167d671
Merge pull request #3882 from MichaelEischer/sftp-init-single-connection
...
sftp: Only connect once to server during `init`
2022-08-26 21:13:28 +02:00
MichaelEischer
3822ded0b3
Merge pull request #3877 from MichaelEischer/no-env-in-help
...
Do not include the actual values of environment variables in help output
2022-08-26 20:59:54 +02:00
Michael Eischer
cf0a8d7758
sftp: Only connect once for repository creation
...
This is especially useful if ssh asks for a password or if closing the
initial connection could return an error due to a problematic server
implementation.
2022-08-26 20:50:40 +02:00
Michael Eischer
dd7cd5b9b3
fuse: remove unused context parameter
2022-08-26 20:48:48 +02:00
Michael Eischer
a0c1ae9f90
mount: Correctly return context.Canceled for interrupted syscalls
...
bazil/fuse expects us to return context.Canceled to signal that a
syscall was successfully interrupted. Returning a wrapped version of
that error however causes the fuse library to signal an EIO (input/output
error). Thus unwrap context.Canceled errors before returning them.
2022-08-26 20:48:48 +02:00
Michael Eischer
5d0649faaf
Update help output in docs
2022-08-26 20:44:01 +02:00
Michael Eischer
faa4597af1
Set name for option values of cli
2022-08-26 20:42:34 +02:00
Michael Eischer
6ed157aee6
Do not include the actual values of environment variables in help output
...
This results in printing a `(default: $ENV) (default: value)` suffix for
the corresponding options which looks strange. In addition, some of the
environment variables might contain secrets which should not be
displayed.
2022-08-26 20:39:54 +02:00
MichaelEischer
f7808245aa
Merge pull request #3878 from MichaelEischer/cheaper-cache-load
...
cache: Just try to open cache entry without calling stat first
2022-08-26 20:33:36 +02:00
MichaelEischer
bee15dd555
Merge pull request #3879 from MichaelEischer/mem-optimize
...
Some random (minor) memory-allocation optimizations
2022-08-26 20:33:02 +02:00
MichaelEischer
0e1d082b12
Merge pull request #3886 from MichaelEischer/recommend-s3-over-b2
...
doc: recommend usage of B2's S3 API
2022-08-26 20:29:05 +02:00
Alexander Neumann
d464543171
Update repo version table
2022-08-25 21:30:25 +02:00
Alexander Neumann
6b40456db7
Set development version for 0.14.0
2022-08-25 19:55:05 +02:00
Alexander Neumann
1bc87e1718
Add version for 0.14.0
2022-08-25 19:55:00 +02:00
Alexander Neumann
c1a5da56e3
Update manpages and auto-completion
2022-08-25 19:55:00 +02:00
Alexander Neumann
193c62dfc3
Generate CHANGELOG.md for 0.14.0
2022-08-25 19:54:02 +02:00
Alexander Neumann
a825e0d409
Prepare changelog for 0.14.0
2022-08-25 19:54:01 +02:00
rawtaz
b824d8cdcc
Merge pull request #3891 from MichaelEischer/fix-secondary-repo-typo
...
Fix typo in the environment variable name for `--from-password-file`
2022-08-24 22:38:47 +02:00
Michael Eischer
a2e89234fc
Fix typo in the environment variable name for --from-password-file
2022-08-24 22:25:18 +02:00
MichaelEischer
b4ae05627f
Merge pull request #3876 from MichaelEischer/document-aws-session-token
...
doc: document aws session token
2022-08-24 22:20:26 +02:00
MichaelEischer
bd7bca2b51
Merge pull request #3889 from restic/prepare-0-14
...
Polish changelog entries
2022-08-24 21:28:44 +02:00
MichaelEischer
be90a565cc
Merge pull request #3887 from MichaelEischer/rclone-permanent-error
...
rclone: Return a permanent error if rclone already exited
2022-08-24 21:19:00 +02:00
Michael Eischer
7a5d29ce24
Further changelog polishing
2022-08-24 21:13:14 +02:00
MichaelEischer
d198a77d86
Merge pull request #3888 from restic/doc-prepare-repo
...
doc: Improve/clarify preparing and versions of repositories
2022-08-24 20:40:49 +02:00
Leo R. Lundgren
cb5a61c46e
doc: Improve/clarify preparing and versions of repositories
2022-08-24 02:08:46 +02:00
Leo R. Lundgren
ee6b9dc492
Polish changelog entries
2022-08-24 01:23:51 +02:00
Michael Eischer
506d92e87c
rclone: Return a permanent error if rclone already exited
...
rclone can exit early for example when the connection to rclone is
relayed for example via ssh: `-o rclone.program='ssh user@example.org
forced-command'`
2022-08-23 22:05:04 +02:00
MichaelEischer
ad6eabbfa5
Merge pull request #3883 from MichaelEischer/update-dependencies
...
Update dependencies
2022-08-23 21:44:38 +02:00
Michael Eischer
7681a63fdb
restic: Cleanup xattr error handling for Solaris
...
Since xattr 0.4.8 (https://github.com/pkg/xattr/pull/68 ) returns ENOTSUP
similar to Linux.
2022-08-23 21:25:15 +02:00
Michael Eischer
99e4ccbd94
remain compatible with go 1.15
2022-08-23 21:25:14 +02:00
Michael Eischer
22f46c18f9
downgrade bazil/fuse again to retain macOS support
2022-08-23 21:24:47 +02:00
Michael Eischer
6db979b3a6
update dependencies
2022-08-23 21:24:46 +02:00
MichaelEischer
9cdc8da10f
Merge pull request #3881 from MichaelEischer/cleanup-release-helper
...
Cleanup release helper
2022-08-23 21:10:57 +02:00
MichaelEischer
b51e73e78f
Merge pull request #3884 from MichaelEischer/fix-index-saving-progress
...
repository: Do not report ignored packs in EachByPack
2022-08-23 21:10:07 +02:00
MichaelEischer
98dcd0a887
Merge pull request #3874 from MichaelEischer/disk-wear-note
...
Add note that larger packs increase disk wear
2022-08-21 22:18:08 +02:00
Michael Eischer
e0d6bf525c
doc: fix typo
2022-08-21 19:12:19 +02:00
Michael Eischer
5d0523e2f1
Add note that larger packs increase disk wear
2022-08-21 19:12:19 +02:00
Michael Eischer
c586a5e20f
doc: recommend usage of B2's S3 API
2022-08-21 11:39:03 +02:00
Michael Eischer
623556bab6
b2: Increase list size to maximum
...
Just request as many files as possible in one call to reduce the number
of network roundtrips.
2022-08-21 11:20:03 +02:00
Michael Eischer
de0162ea76
backend/retry: Overwrite failed uploads instead of deleting them
...
For backends which are able to atomically replace files, we just can
overwrite the old copy, if it is necessary to retry an upload. This has
the benefit of issuing one operation less and might be beneficial if a
backend storage, due to bugs or similar, could mix up the order of the
upload and delete calls.
2022-08-21 11:14:53 +02:00
Michael Eischer
fc506f8538
b2: Repeat deleting until all file versions are removed
...
When hard deleting the latest file version on B2, this uncovers earlier
versions. If an upload required retries, multiple version might exist
for a file. Thus to reliably delete a file, we have to remove all
versions of it.
2022-08-21 11:11:00 +02:00
Michael Eischer
cc4728d287
repository: Do not report ignored packs in EachByPack
...
Ignored packs were reported as an empty pack by EachByPack. The most
immediate effect of this is that the progress bar for rebuilding the
index reports processing more packs than actually exist.
2022-08-21 10:38:40 +02:00
Michael Eischer
c3374b3ea5
helper: download modules as first step
...
There's no use in running that step in parallel.
2022-08-20 12:11:54 +02:00
Michael Eischer
7f0929e519
helper: Reduce number of parallel builds a bit
...
The go compiler is already parallelized. The high concurrency caused my
podman container to hit a resource limit.
2022-08-20 12:10:48 +02:00
Michael Eischer
ed94678820
helper: cleanups
2022-08-20 12:10:29 +02:00
Michael Eischer
e530d422a0
helper: don't setup cmd paths twice
2022-08-20 12:09:42 +02:00
Michael Eischer
7a992fc794
repository: Reduce buffer reallocations in ForAllIndexes
...
Previously the buffer was grown incrementally inside `repo.LoadUnpacked`.
But we can do better as we already know how large the index will be.
Allocate a bit more memory to increase the chance that the buffer can be
reused in the future.
2022-08-19 21:13:40 +02:00
Michael Eischer
77b1980d8e
repository: MasterIndex.Packs: reduce allocations
2022-08-19 21:10:43 +02:00
Michael Eischer
6ff9517e45
repository: MasterIndex.ListPacks / Index.EachByPack allow earlier GC
...
Allow earlier garbage collection of some of the intermediate data
structures.
2022-08-19 21:06:33 +02:00
Michael Eischer
ce902aac67
cache: Just try to open cache entry without calling stat first
...
Instead of first checking whether a file is in the repository cache and
then opening it, we just can open the file. This saves one stat call. If
the file is in the cache, everything is fine and otherwise the code
follows its normal fallback path.
2022-08-19 20:59:06 +02:00
Michael Eischer
b6c86ababe
doc: document aws session token
2022-08-19 20:41:15 +02:00
MichaelEischer
0d9ac78437
Merge pull request #3873 from MichaelEischer/gofmt-comments
...
gofmt comments
2022-08-19 19:54:30 +02:00
MichaelEischer
7e96a5af62
Merge pull request #3872 from MichaelEischer/fuse-fix
...
mount: Only remember successful snapshot refreshes
2022-08-19 19:21:29 +02:00
Michael Eischer
f414db987d
gofmt all files
...
Apparently the rules for comment formatting have changed with go 1.19.
2022-08-19 19:12:26 +02:00
Michael Eischer
522406b4f0
mount: Only remember successful snapshot refreshes
...
If the context provided by the fuse library is canceled before the index
was loaded this could lead to missing snapshots.
2022-08-19 19:07:07 +02:00
MichaelEischer
dbca93da28
Merge pull request #3742 from MichaelEischer/from-repo
...
copy: replace --repo2 with --from-repo
2022-08-19 19:01:04 +02:00
MichaelEischer
b4dfab002a
Merge pull request #3691 from greatroar/tag-filenames
...
Sanitize tags when used as filenames by restic mount
2022-08-19 18:33:44 +02:00
Michael Eischer
2758d76b77
copy: replace --repo2 with --from-repo
...
`init` and `copy` use `--repo2` with two different meaning which has
proven to be confusing for users. `--from-repo` now consistently marks a
source repository from which data is read. `--repo` is now always the
target/destination repository.
2022-08-19 18:33:26 +02:00
Michael Eischer
af50fe9ac0
mount: Map slashes in tags to underscores
...
Suggested-by: greatroar <>
2022-08-19 18:17:57 +02:00
MichaelEischer
4cccffab58
Merge pull request #3862 from restic/3861-forget-invalid-policy
...
forget: Error when invalid unit is given in duration policy
2022-08-18 20:45:26 +02:00
Michael Eischer
2ea6c82cf6
comment cleanup
...
gofmt reformatted the comment
2022-08-18 20:15:38 +02:00
Michael Eischer
bb27f7408c
forget: Fail test if duration parsing error is missing
2022-08-18 20:14:09 +02:00
MichaelEischer
c4b3a154ba
Merge pull request #3865 from restic/rawtaz-doc-comp-typo
...
doc: Fix typo in compression section
2022-08-18 19:37:44 +02:00
rawtaz
2cb2aa31cd
doc: Fix typo in compression section
2022-08-12 22:17:38 +02:00
Leo R. Lundgren
6f517858e8
forget: Error when invalid unit is given in duration policy
2022-08-10 13:37:26 +02:00
rawtaz
f0bb4f8708
Merge pull request #3857 from restic/rawtaz-gcs-account
...
doc: Update links to GCS documentation
2022-08-08 22:52:30 +02:00
Michael Eischer
40c8755b13
doc: Update more links to GCS documentation
2022-08-08 20:00:44 +02:00
rawtaz
f673068dbb
doc: Update link to GCS documentation
...
Updates the link to Google Cloud Storage documentation about creating a service account key.
2022-08-08 12:51:43 +02:00
rawtaz
f26231c9e6
Merge pull request #3852 from MichaelEischer/snapshots-processed
...
stats: Add snapshots count to json output
2022-08-08 00:05:15 +02:00
MichaelEischer
80e3efffef
Merge pull request #3855 from mattxtaz/master
...
Fix typo with double percentage in help text
2022-08-07 22:44:36 +02:00
mattxtaz
01ab36336f
Fix typo with double percentage in help text
2022-08-07 20:21:05 +01:00
Michael Eischer
6e92d852a8
stats: Add snapshots count to json output
2022-08-07 15:44:09 +02:00
MichaelEischer
9ad3ad5972
Merge pull request #3850 from lbausch/go1.19
...
Update tests to Go 1.19
2022-08-07 14:56:17 +02:00
MichaelEischer
2930a102de
Merge pull request #3731 from metalsp0rk/feature/min-packsize-flag
...
Feature: min packsize flag
2022-08-07 14:54:45 +02:00
MichaelEischer
8fa64a8f99
Merge pull request #3036 from greatroar/refactor-fuse
...
Clean up internal/fuse
2022-08-07 14:46:30 +02:00
Michael Eischer
f3fdc66b32
restic: Use stable sorting in snapshot policy
...
sort.Sort is not guaranteed to be stable. Go 1.19 has changed the
sorting algorithm which resulted in changes of the sort order. When
comparing snapshots with identical timestamp but different paths and
tags lists, there is not meaningful order among them. So just keep their
order stable.
2022-08-07 14:10:40 +02:00
Lorenz Bausch
b82f4824f0
Bump golangci-lint version
2022-08-07 14:10:39 +02:00
Lorenz Bausch
0b9b4c52ad
Update tests to Go 1.19
2022-08-07 14:10:39 +02:00
Michael Eischer
0b7291b8b2
mount: Fix parent inode used by snapshots dir
2022-08-07 13:03:32 +02:00
greatroar
cfa80e2c6b
mount: remove unused inode field from root node
2022-08-07 13:03:26 +02:00
MichaelEischer
74ae76036f
Merge pull request #2913 from aawsome/mount-snapshot-slashes
...
mount: Make snapshots dir structure customizable
2022-08-07 12:27:59 +02:00
MichaelEischer
09497aec02
Merge pull request #3826 from MichaelEischer/debug-log-for-release
...
Debug log for release build
2022-08-07 12:15:27 +02:00
Michael Eischer
83b4c50ee3
Mention --snapshot-template and --time-template in changelog
2022-08-07 12:13:06 +02:00
Michael Eischer
caa17988a3
fuse: Redesign snapshot dirstruct
...
Cleanly separate the directory presentation and the snapshot directory
structure. SnapshotsDir now translates the dirStruct into a format
usable by the fuse library and contains only minimal special case rules.
All decisions have moved into SnapshotsDirStructure which now creates a
fully preassembled tree data structure.
2022-08-07 12:13:06 +02:00
Michael Eischer
a3e48da3a3
Add changelog for DEBUG_LOG available in release builds
2022-08-05 23:49:39 +02:00
Michael Eischer
b3cdee66a9
update documentation to reflect DEBUG_LOG for release builds
2022-08-05 23:49:39 +02:00
Michael Eischer
1ed775e3a8
debug: support roundtripper logging also for release builds
...
Different from debug builds do not use the eofDetectRoundTripper if
logging is disabled.
2022-08-05 23:49:39 +02:00
Michael Eischer
38becfc436
debug: enable debug support for release builds
2022-08-05 23:49:39 +02:00
Michael Eischer
82c268c917
Remove unused hooks mechanism
2022-08-05 23:49:39 +02:00
Michael Eischer
7266f07c87
repository: StreamPack in parts if there are too large gaps
...
For large pack sizes we might be only interested in the first and last
blob of a pack file. Thus stream a pack file in multiple parts if the
gaps between requested blobs grow too large.
2022-08-05 23:48:36 +02:00
Michael Eischer
55a11c1396
Reword prune --repack-small description
2022-08-05 23:48:36 +02:00
Michael Eischer
eaf43607f9
Add note that pack-size is not an exact limit
2022-08-05 23:48:36 +02:00
Michael Eischer
7f3b2be1e8
s3: Disable multipart uploads below 200MB
2022-08-05 23:48:36 +02:00
Michael Eischer
176b387d98
Always repack very small pack files
2022-08-05 23:48:36 +02:00
Michael Eischer
324935cb80
Only repack small files if there are multiple of them
2022-08-05 23:48:34 +02:00
Michael Eischer
1b076cda97
rename option to --pack-size
2022-08-05 23:47:43 +02:00
Michael Eischer
d7e2892048
Add changelog for packsize option
2022-08-05 23:47:43 +02:00
Michael Eischer
8a44258b6f
update restic help snippets in documentation
2022-08-05 23:47:43 +02:00
Michael Eischer
420ddc03c9
rework pack size parameter documentation
2022-08-05 23:47:43 +02:00
Kyle Brennan
e43be84eb8
document minPackSize
2022-08-05 23:47:41 +02:00
Kyle Brennan
1e3f05c3f1
repository: prevent header overfill
2022-08-05 23:47:12 +02:00
Michael Eischer
6a6d313c9a
prune: reduce priority of repacking small packs
2022-08-05 23:47:12 +02:00
Kyle Brennan
0269381b8d
prune: add repack-small parameter
2022-08-05 23:47:12 +02:00
Michael Eischer
0a6fa602c8
add option for setting min pack size
2022-08-05 23:47:12 +02:00
Michael Eischer
2db7733ee3
fuse: remove unused MetaDir
2022-08-05 23:46:46 +02:00
Michael Eischer
f678f7cb04
fuse: cleanup test
2022-08-05 23:46:46 +02:00
Alexander Weiss
1751afae26
Make snapshots dirs in mount command customizable
2022-08-05 23:46:46 +02:00
Alexander Weiss
57f4003f2f
Generalize fuse snapshot dirs implemetation
...
+ allow "/" in tags and snapshot template
2022-08-05 23:46:46 +02:00
Alexander Weiss
696c18e031
Add possibility to set snapshot ID (used in test)
2022-08-05 23:46:46 +02:00
MichaelEischer
04a8ee80fb
Merge pull request #3829 from MichaelEischer/prune-refactor
...
Split prune into slightly small functions
2022-08-05 23:29:52 +02:00
MichaelEischer
15679be858
Merge pull request #3841 from MichaelEischer/compression-env
...
Add environment variable RESTIC_COMPRESSION
2022-08-05 23:20:04 +02:00
MichaelEischer
7d14b1baf1
Merge pull request #3845 from greatroar/solaris-xattr
...
internal/restic: Handle EINVAL for xattr on Solaris
2022-08-04 22:04:52 +02:00
greatroar
ad6ac680af
internal/restic: Handle EINVAL for xattr on Solaris
...
Also make the errors a bit less verbose by not prepending the operation,
since pkg/xattr already does that. Old errors looked like
Listxattr: xattr.list /myfiles/.zfs/snapshot: invalid argument
2022-08-01 12:45:17 +02:00
MichaelEischer
846d021db5
Merge pull request #3840 from greatroar/sftp-init
...
Speed up restic init over slow SFTP links
2022-07-31 19:54:59 +02:00
greatroar
b9fa6e05bd
Add changelog for #3837/#3840
2022-07-30 22:53:52 +02:00
Michael Eischer
73053674d9
repository: Test fallback to existing blobs
2022-07-30 17:37:07 +02:00
Michael Eischer
e85a21eda2
prune: move code
2022-07-30 17:37:07 +02:00
Michael Eischer
623770eebb
repository: try to recover from invalid blob while repacking
...
If a blob that should be kept is invalid, Repack will now try to request
the blob using LoadBlob. Only return an error if that fails.
2022-07-30 17:37:07 +02:00
Michael Eischer
d0590b7841
prune: Add internal integrity check
...
After repacking every blob that should be kept must have been repacked.
We have seen a few cases in which a single blob went missing, which
could have been caused by a bitflip somewhere. This sanity check might
help catch some of these cases.
2022-07-30 17:37:07 +02:00
Michael Eischer
5cbde03eae
prune: split into smaller functions
2022-07-30 17:37:07 +02:00
Alexander Weiss
7643237da5
prune: separate collecting/printing/pruning
2022-07-30 17:37:07 +02:00
Michael Eischer
5723c72eb1
Add environment variable RESTIC_COMPRESSION
2022-07-30 16:21:53 +02:00
greatroar
2bdc40e612
Speed up restic init over slow SFTP links
...
pkg/sftp.Client.MkdirAll(d) does a Stat to determine if d exists and is
a directory, then a recursive call to create the parent, so the calls
for data/?? each take three round trips. Doing a Mkdir first should
eliminate two round trips for 255/256 data directories as well as all
but one of the top-level directories.
Also, we can do all of the calls concurrently. This may reintroduce some
of the Stat calls when multiple goroutines try to create the same
parent, but at the default number of connections, that should not be
much of a problem.
2022-07-30 13:09:08 +02:00
greatroar
23ebec717c
Remove stale comments from backend/sftp
...
The preExec and postExec functions were removed in
0bdb131521
from 2018.
2022-07-30 13:07:25 +02:00
MichaelEischer
4ffd479ba4
Merge pull request #3773 from MichaelEischer/efficient-dir-json
...
Reduce memory usage for large directories/files
2022-07-23 17:47:32 +02:00
Michael Eischer
2ba14160de
Add changelog for the optimized tree serialization
2022-07-23 14:49:08 +02:00
Michael Eischer
4a10ebed15
archiver: reduce memory usage for large files
...
FutureBlob now uses a Take() method as a more memory-efficient way to
retrieve the futures result. In addition, futures are now collected
while saving the file. As only a limited number of blobs can be queued
for uploading, for a large file nearly all FutureBlobs already have
their result ready, such that the FutureBlob object just consumes
memory.
2022-07-23 14:45:07 +02:00
Michael Eischer
b817681a11
archiver: Incrementally serialize tree nodes
...
That way it is not necessary to keep both the Nodes forming a Tree and
the serialized JSON version in memory.
2022-07-23 14:45:07 +02:00
Michael Eischer
c206a101a3
archiver: unify FutureTree/File into futureNode
...
There is no real difference between the FutureTree and FutureFile
structs. However, differentiating both increases the size of the
FutureNode struct.
The FutureNode struct is now only 16 bytes large on 64bit platforms.
That way is has a very low overhead if the corresponding file/directory
was not processed yet.
There is a special case for nodes that were reused from the parent
snapshot, as a go channel seems to have 96 bytes overhead which would
result in a memory usage regression.
2022-07-23 14:45:07 +02:00
Michael Eischer
32f4997733
archiver: remove unused fileInfo from progress callback
2022-07-23 14:16:23 +02:00
Michael Eischer
dcb00fd2d1
archiver: cleanup Saver interface
2022-07-23 14:16:23 +02:00
Michael Eischer
79321a195c
archiver: remove dead attribute from FutureNode
2022-07-23 14:16:23 +02:00
MichaelEischer
049f4c4144
Merge pull request #3730 from MichaelEischer/stricter-check
...
Let `check` warn about legacy variants of the repo format version 1
2022-07-23 14:14:50 +02:00
Michael Eischer
3bf53da672
Add changelog for stricter checks
2022-07-23 11:21:26 +02:00
Michael Eischer
5a6f2f9fa0
Fix S3 legacy layout migration
2022-07-23 11:19:32 +02:00
Michael Eischer
04e49924fb
checker: Fix S3 legacy layout detection
2022-07-23 11:19:32 +02:00
Michael Eischer
768c890fcb
check: Deprecate `--check-unused`
...
Unused blobs are not a problem but rather expected to exist now that
prune by default does not remove every unused blob. However, the option
has caused questions from users whether a repository is damaged or not,
so just remove that option.
Note that the remaining code is left intact as it is still useful for
our test cases.
2022-07-23 11:19:32 +02:00
Michael Eischer
fcb3ddf181
check: Complain about usage of s3 legacy layout
2022-07-23 11:19:32 +02:00
Michael Eischer
8b8bd4e8ac
check: complain about mixed pack files
2022-07-23 11:19:32 +02:00
MichaelEischer
443cc49afd
Merge pull request #3830 from MichaelEischer/cleanup-repo
...
Extract Load/SaveTree/JSONUnpacked from repository
2022-07-23 10:46:13 +02:00
MichaelEischer
1f5369e072
Merge pull request #3831 from MichaelEischer/move-code
...
Move code out of the restic package and consolidate backend specific code
2022-07-23 10:33:05 +02:00
MichaelEischer
827ab02eea
Merge pull request #3661 from rgammans/azure_sas_support
...
add SAS authentication option for Azure repos
2022-07-23 10:32:03 +02:00
MichaelEischer
e9c39442fb
Merge pull request #3827 from MichaelEischer/backup-doc
...
backup: clarify usage string
2022-07-23 10:31:33 +02:00
Michael Eischer
9729e6d7ef
backend: extract readerat from restic package
2022-07-17 15:29:09 +02:00
Michael Eischer
c44b21d366
restorer: extract hardlinks index from restic package
2022-07-17 13:45:42 +02:00
Michael Eischer
8c11fc3ec9
crypto: move crypto buffer helpers
2022-07-17 13:42:23 +02:00
Michael Eischer
a0cef9f247
limiter: move to internal/backend
2022-07-17 13:40:15 +02:00
Michael Eischer
163ab9c025
mock: move to internal/backend
2022-07-17 13:40:06 +02:00
Michael Eischer
89d3ce852b
repository: extract Load/StoreJSONUnpacked
...
A Load/Store method for each data type is much clearer. As a result the
repository no longer needs a method to load / store json.
2022-07-17 13:22:00 +02:00
Michael Eischer
fbcbd5318c
repository: extract LoadTree/SaveTree
...
The repository has no real idea what a Tree is. So these methods never
belonged there.
2022-07-17 13:11:28 +02:00
MichaelEischer
d9ea1e9ee2
Merge pull request #3290 from aawsome/prune-handle-duplicates
...
prune: Handle duplicate blobs more efficiently
2022-07-17 11:51:54 +02:00
Michael Eischer
715d457aad
prune: code cleanups
2022-07-17 11:41:56 +02:00
Michael Eischer
9be1bd2acc
prune: handle very high duplication of some blobs
...
Suggested-By: Alexander Weiss <alex@weissfam.de>
2022-07-17 11:39:56 +02:00
Alexander Weiss
7478cbf70e
prune: Enhance treatment of duplicates
2022-07-17 00:22:23 +02:00
Michael Eischer
b2043e8198
backup: clarify usage string
...
Using the `--files-from` options it is possible to run `backup` without
specifying any source paths directly on the command line.
2022-07-17 00:03:22 +02:00
Michael Eischer
5639c41b6a
azure: Strip ? prefix from sas token
2022-07-16 23:55:18 +02:00
Roger Gammans
64a7ec5341
azure: add SAS authentication option
2022-07-16 23:55:18 +02:00
MichaelEischer
6cbeb4a9f9
Merge pull request #3825 from restic/rawtaz-doc-gdrive
...
doc: Add note about using rclone for Google Drive
2022-07-16 19:21:59 +02:00
rawtaz
f5c219f5a2
doc: Add note about using rclone for Google Drive
...
It wasn't clear that Google Cloud Storage and Google Drive are two different services and that one should use the rclone backend for the latter. This commit adds a note with this information.
2022-07-16 13:22:38 +02:00
MichaelEischer
d71b29221b
Merge pull request #3822 from JsBergbau/doc-max-compression
...
Added hint for --compression max in migration process
2022-07-12 23:02:45 +02:00
Michael Eischer
71ff6b77f0
doc: Rework hint to repack with max compression
2022-07-12 21:24:40 +02:00
MichaelEischer
6970d05d47
Merge pull request #3741 from lbausch/repository-wording
...
Wording: change repo to repository
2022-07-12 21:15:33 +02:00
MichaelEischer
3934480da4
Merge pull request #3819 from lbausch/restore-validate-patterns
...
restore: validate include/exclude patterns
2022-07-12 20:58:41 +02:00
MichaelEischer
71a0157c2c
Merge pull request #3820 from lbausch/patch-1
...
Fix wording in changelog template
2022-07-12 20:51:52 +02:00
Michael Eischer
2aad6f24b5
doc: update sample help output
2022-07-12 20:49:01 +02:00
Michael Eischer
ec4dfa3c66
Wording: replace further repo occurrences with repository
2022-07-12 20:48:01 +02:00
JsBergbau
8d3f04aefa
Added hint for --compression max in migration process
...
Added hint for --compression max in migration process. Since this is a onetime process users should be aware of this and consider this step.
2022-07-12 00:08:26 +02:00
Lorenz Bausch
b609523582
Add changelog entry
2022-07-09 22:24:56 +02:00
lbausch
ac96a4138d
Fix wording in changelog template
2022-07-09 22:13:54 +02:00
Lorenz Bausch
7e36ec279d
Test restore fails when using invalid patterns
2022-07-08 20:09:26 +02:00
Lorenz Bausch
be524f0b78
Add testRunRestoreAssumeFailure function
2022-07-08 20:09:25 +02:00
Lorenz Bausch
9a7db6675c
Restore: validate provided patterns
2022-07-08 20:09:25 +02:00
Lorenz Bausch
d6e3c7f28e
Wording: change repo to repository
2022-07-08 20:05:35 +02:00
MichaelEischer
98a3125ce4
Merge pull request #3816 from mattxtaz/master
...
Fix minor typo in docs
2022-07-05 21:30:26 +02:00
mattxtaz
87d899c099
Fix minor typo in docs
2022-07-05 20:06:27 +01:00
Jerome Küttner
6f3883c9d2
add changelog
2022-07-05 08:47:48 +02:00
Alexander Neumann
545220803b
Merge pull request #3813 from MichaelEischer/fix-blob-saver-data-race
...
Fix data race in blob_saver
2022-07-04 08:10:01 +02:00
Michael Eischer
ce89018902
Fix data race in blob_saver
...
After the `BlobSaver` job is submitted, the buffer can be released and
reused by another `FileSaver` even before `BlobSaver.Save` returns. That
FileSaver will the change `buf.Data` leading to wrong backup statistics.
Found by `go test -race ./...`:
WARNING: DATA RACE
Write at 0x00c0000784a0 by goroutine 41:
github.com/restic/restic/internal/archiver.(*FileSaver).saveFile()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:176 +0x789
github.com/restic/restic/internal/archiver.(*FileSaver).worker()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:242 +0x2af
github.com/restic/restic/internal/archiver.NewFileSaver.func2()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:88 +0x5d
golang.org/x/sync/errgroup.(*Group).Go.func1()
/home/michael/go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x91
Previous read at 0x00c0000784a0 by goroutine 29:
github.com/restic/restic/internal/archiver.(*BlobSaver).Save()
/home/michael/Projekte/restic/restic/internal/archiver/blob_saver.go:57 +0x1dd
github.com/restic/restic/internal/archiver.(*BlobSaver).Save-fm()
<autogenerated>:1 +0xac
github.com/restic/restic/internal/archiver.(*FileSaver).saveFile()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:191 +0x855
github.com/restic/restic/internal/archiver.(*FileSaver).worker()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:242 +0x2af
github.com/restic/restic/internal/archiver.NewFileSaver.func2()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:88 +0x5d
golang.org/x/sync/errgroup.(*Group).Go.func1()
/home/michael/go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x91
2022-07-03 14:47:53 +02:00
MichaelEischer
b6a38d43b3
Merge pull request #3611 from MichaelEischer/auto-concurrency
...
Adjust worker goroutines to number of backend connections
2022-07-03 12:33:32 +02:00
Michael Eischer
3af9c2cc58
Document automatic CPU/IO-concurrency
2022-07-03 12:19:26 +02:00
Michael Eischer
6f53ecc1ae
adapt workers based on whether an operation is CPU or IO-bound
...
Use runtime.GOMAXPROCS(0) as worker count for CPU-bound tasks,
repo.Connections() for IO-bound task and a combination if a task can be
both. Streaming packs is treated as IO-bound as adding more worker
cannot provide a speedup.
Typical IO-bound tasks are download / uploading / deleting files.
Decoding / Encoding / Verifying are usually CPU-bound. Several tasks are
a combination of both, e.g. for combined download and decode functions.
In the latter case add both limits together. As the backends have their
own concurrency limits restic still won't download more than
repo.Connections() files in parallel, but the additional workers can
decode already downloaded data in parallel.
2022-07-03 12:19:26 +02:00
MichaelEischer
cd50feb66f
Merge pull request #3489 from MichaelEischer/async-pack-uploads
...
Asynchronously upload pack files
2022-07-03 11:56:05 +02:00
Michael Eischer
74df9d5998
Add changelog for async pack uploads
2022-07-03 11:34:01 +02:00
Michael Eischer
dbb5860dc9
Document connections and compression option
2022-07-03 11:19:24 +02:00
Michael Eischer
753e56ee29
repository: Limit to a single pending pack file
...
Use only a single not completed pack file to keep the number of open and
active pack files low. The main change here is to defer hashing the pack
file to the upload step. This prevents the pack assembly step to become
a bottleneck as the only task is now to write data to the temporary pack
file.
The tests are cleaned up to no longer reimplement packer manager
functions.
2022-07-02 22:42:34 +02:00
Michael Eischer
fa25d6118e
archiver: Reduce tree saver concurrency
...
Large amount of tree savers have no obvious benefit, however they can
increase the amount of (potentially large) trees kept in memory.
2022-07-02 22:42:34 +02:00
Michael Eischer
bba1e81719
archiver: Limit blob saver count to GOMAXPROCS
...
Now with the asynchronous uploaders there's no more benefit from using
more blob savers than we have CPUs. Thus use just one blob saver for
each CPU we are allowed to use.
2022-07-02 22:42:34 +02:00
Michael Eischer
120ccc8754
repository: Rework blob saving to use an async pack uploader
...
Previously, SaveAndEncrypt would assemble blobs into packs and either
return immediately if the pack is not yet full or upload the pack file
otherwise. The upload will block the current goroutine until it
finishes.
Now, the upload is done using separate goroutines. This requires changes
to the error handling. As uploads are no longer tied to a SaveAndEncrypt
call, failed uploads are signaled using an errgroup.
To count the uploaded amount of data, the pack header overhead is no
longer returned by `packer.Finalize` but rather by
`packer.HeaderOverhead`. This helper method is necessary to continue
returning the pack header overhead directly to the responsible call to
`repository.SaveBlob`. Without the method this would not be possible,
as packs are finalized asynchronously.
2022-07-02 22:42:34 +02:00
Michael Eischer
bba4c69a2a
tag: Remove unnecessary flush call
2022-07-02 22:42:23 +02:00
MichaelEischer
3e1de52e0a
Merge pull request #3805 from greatroar/global
...
cmd/restic, limiter: Move config knowledge to internal packages
2022-07-02 21:56:35 +02:00
MichaelEischer
621023a50b
Merge pull request #3772 from MichaelEischer/fix-mixed-index
...
rebuild-index: correctly rebuild index for mixed packs
2022-07-02 20:10:02 +02:00
MichaelEischer
90e9c5c4cc
Merge pull request #3729 from MichaelEischer/full-ids-in-check
...
Include full IDs in check output
2022-07-02 20:09:39 +02:00
MichaelEischer
7137034517
Merge pull request #3811 from restic/fix-secret-string-crash
...
Don't crash if SecretString is uninitialized
2022-07-02 19:56:36 +02:00
Michael Eischer
cdaf9b4f26
Don't crash if SecretString is uninitialized
2022-07-02 19:44:28 +02:00
Michael Eischer
5e0f1c3cef
check: remove dead code
2022-07-02 19:28:57 +02:00
Michael Eischer
0df022fa6d
check: Print full ids
...
The short ids are not always unique. In addition, recovering from
damages is easier when having the full ids as that makes it easier to
access the corresponding files.
2022-07-02 19:28:57 +02:00
Michael Eischer
04c23fa95d
rebuild-index: correctly rebuild index for mixed packs
...
For mixed packs, data and tree blobs were stored in separate index
entries. This results in warning from the check command and maybe other
problems.
2022-07-02 19:24:02 +02:00
MichaelEischer
bb5f196b09
Merge pull request #3733 from restic/improve-stats
...
Improve stats
2022-07-02 19:07:31 +02:00
MichaelEischer
c16f989d4a
Merge pull request #3470 from MichaelEischer/sanitize-debug-log
...
Sanitize debug log
2022-07-02 19:00:54 +02:00
Michael Eischer
00d7fcff96
extend compression feature changelog entry
2022-07-02 18:55:59 +02:00
Michael Eischer
a6e9e08034
Account for pack header overhead at each entry
...
This will miss the pack header crypto overhead and the length field,
which only amount to a few bytes per pack file.
2022-07-02 18:55:58 +02:00
Michael Eischer
856d5e4303
stats: return storage size for raw-data mode
...
raw-data summed up the size of the blob plaintexts. However, with
compression this makes little sense as the storage size in the
repository is lower due to compression. Thus sum up the actual size each
blob takes in the repository.
2022-07-02 18:55:12 +02:00
Alexander Neumann
6c4ceaf1e7
Print number of bytes added to the repo
...
This includes optional compression and crypto overhead.
2022-07-02 18:55:12 +02:00
Alexander Neumann
99634c0936
Return real size from SaveBlob
2022-07-02 18:55:12 +02:00
MichaelEischer
fdc53a9d32
Merge pull request #3787 from MichaelEischer/refactor-repository
...
repository: (Mostly) index-related cleanups
2022-07-02 18:54:04 +02:00
Michael Eischer
6923353c43
redact swift auth token in debug output
2022-07-02 18:47:35 +02:00
Michael Eischer
5a11d14082
redacted keys/token in backend config debug log
2022-07-02 18:47:35 +02:00
Michael Eischer
0936d864a4
redact http authorization header in debug log output
2022-07-02 18:47:35 +02:00
Michael Eischer
ec7c9ce88b
drop unused repository.Loader interface
2022-07-02 18:39:59 +02:00
Michael Eischer
2cd7e90ad1
repository: cleanup
2022-07-02 18:39:59 +02:00
Michael Eischer
c1a8fa4290
repository: remove unused packIDToIndex field
2022-07-02 18:39:59 +02:00
Michael Eischer
e68c3a4e62
repository: simplify CreateIndexFromPacks
2022-07-02 18:39:59 +02:00
Michael Eischer
1974ad7ce2
repository: hide MasterIndex.FinalizeFullIndexes / FinalizeNotFinalIndexes
2022-07-02 18:39:59 +02:00
Michael Eischer
ef53ca4a5a
repository: remove MasterIndex.All()
2022-07-02 18:39:59 +02:00
Michael Eischer
bf81bf0795
repository: Properly set id for finalized index
...
As MergeFinalIndex and index uploads can occur concurrently, it is
necessary for MergeFinalIndex to check whether the IDs for an index were
already set before merging it. Otherwise, we'd loose the ID of an index
which is set _after_ uploading it.
2022-07-02 18:39:59 +02:00
Michael Eischer
e0a7852b8b
repository: remove unused (Master)Index.Count
2022-07-02 18:39:58 +02:00
Michael Eischer
8ef2968f28
repository: remove unused index.ListPack
2022-07-02 18:39:12 +02:00
Michael Eischer
e4f20dea61
repository: inline index.encode
2022-07-02 18:39:12 +02:00
Michael Eischer
fe5a8e137a
repository: remove unused index.Store
2022-07-02 18:39:12 +02:00
Michael Eischer
628ae799ca
repository: make flushPacks private
2022-07-02 18:39:12 +02:00
Michael Eischer
ed8aa15376
repository: add Save method to MasterIndex interface
2022-07-02 18:38:56 +02:00
Michael Eischer
a77d5c4d11
repository: index saving belongs into the MasterIndex
2022-07-02 18:38:56 +02:00
MichaelEischer
19641bf828
Merge pull request #3810 from greatroar/revert-3786
...
Revert "restic prune: Merge three loops over the index"
2022-07-01 23:13:39 +02:00
greatroar
a0fa9c6e9f
Revert "restic prune: Merge three loops over the index"
...
This reverts commit 8bdfcf779f
.
Should fix #3809 . Also needed to make #3290 apply cleanly.
2022-06-30 15:27:34 +02:00
greatroar
90d2c0502b
cmd/restic, limiter: Move config knowledge to internal packages
...
The GlobalOptions struct now embeds a backend.TransportOptions, so it
doesn't need to construct one in open and create. The upload and
download limits are similarly now a struct in internal/limiter that is
embedded in GlobalOptions.
2022-06-22 18:29:58 +02:00
MichaelEischer
bc96879d41
Merge pull request #3785 from MichaelEischer/replace-tomb-usage
...
Remove usage of tomb package
2022-06-19 14:42:48 +02:00
MichaelEischer
307f14604f
Merge pull request #3795 from greatroar/sema
...
backend: Move semaphores to a dedicated package
2022-06-18 17:12:01 +02:00
MichaelEischer
19581dbc18
Merge pull request #3786 from greatroar/prune
...
restic prune: Merge three loops over the index
2022-06-18 16:54:50 +02:00
greatroar
8bdfcf779f
restic prune: Merge three loops over the index
...
There were three loops over the index in restic prune, to find
duplicates, to determine sizes (in pack.Size) and to generate packInfos.
These three are now one loop. This way, prune doesn't need to construct
a set of duplicate blobs, pack.Size doesn't need to contain special
logic for prune's use case (the onlyHdr argument) and pack.Size doesn't
need to construct a map only to have it immediately transformed into a
different map.
Some quick testing on a 160GiB local repo doesn't show running time or
memory use of restic prune --dry-run changing significantly.
2022-06-18 10:40:33 +02:00
greatroar
910d917b71
backend: Move semaphores to a dedicated package
...
... called backend/sema. I resisted the temptation to call the main
type sema.Phore. Also, semaphores are now passed by value to skip a
level of indirection when using them.
2022-06-18 10:01:58 +02:00
MichaelEischer
2c893fe43c
Merge pull request #3798 from greatroar/errors
...
all: Move away from pkg/errors, easy cases
2022-06-17 19:01:40 +02:00
greatroar
f92ecf13c9
all: Move away from pkg/errors, easy cases
...
github.com/pkg/errors is no longer getting updates, because Go 1.13
went with the more flexible errors.{As,Is} function. Use those instead:
errors from pkg/errors already support the Unwrap interface used by 1.13
error handling. Also:
* check for io.EOF with a straight ==. That value should not be wrapped,
and the chunker (whose error is checked in the cases changed) does not
wrap it.
* Give custom Error methods pointer receivers, so there's no ambiguity
when type-switching since the value type will no longer implement error.
* Make restic.ErrAlreadyLocked private, and rename it to
alreadyLockedError to match the stdlib convention that error type
names end in Error.
* Same with rest.ErrIsNotExist => rest.notExistError.
* Make s3.Backend.IsAccessDenied a private function.
2022-06-14 08:36:38 +02:00
MichaelEischer
0c0e7b6957
Merge pull request #3776 from wjiec/bugfix/maxkeys-in-search
...
Limit number of key files tested while opening a repository
2022-06-12 15:51:10 +02:00
Michael Eischer
c9ef873192
tweak password test count changelog
2022-06-12 15:39:06 +02:00
Michael Eischer
d08549f0ae
fix flaky key test
2022-06-12 14:19:06 +02:00
Jayson Wang
f144920ed5
fix handling of maxKeys in SearchKey
2022-06-12 14:19:06 +02:00
Alexander Neumann
1dd4b9b60e
Merge pull request #3788 from greatroar/sftp-posix-rename
...
backend/sftp: Support atomic rename
2022-06-06 19:39:48 +02:00
Alexander Neumann
07114ccb21
Merge pull request #3789 from greatroar/fix-loadblob
...
internal/repository: Fix LoadBlob + fuzz test
2022-06-06 19:33:42 +02:00
greatroar
c9557b2822
internal/repository: Fix LoadBlob + fuzz test
...
When given a buf that is big enough for a compressed blob but not its
decompressed contents, the copy at the end of LoadBlob would skip the
last part of the contents.
Fixes #3783 .
2022-06-06 17:02:28 +02:00
greatroar
fa8f02292e
backend/sftp: Support atomic rename
...
... if the server has posix-rename@openssh.com .
OpenSSH introduced this extension in 2008:
7c29661471
2022-06-06 13:40:42 +02:00
Alexander Neumann
7d64aa7f57
Merge pull request #3784 from MichaelEischer/cleanup-migrate-check
...
migrate: Cleanup option to request repository check
2022-06-05 17:37:45 +02:00
Michael Eischer
853ceb3bec
get rid of tomb package
2022-06-05 15:54:57 +02:00
Michael Eischer
e002b09d57
archiver: free workers once finished
2022-06-05 15:48:10 +02:00
Michael Eischer
408ac1a0c2
archiver: remove tomb usage
2022-06-05 15:47:52 +02:00
Michael Eischer
5eba1217e7
migrate: Cleanup option to request repository check
2022-06-04 23:45:00 +02:00
MichaelEischer
0cb6b3d80a
Merge pull request #3778 from greatroar/ellipsis
...
cmd/restic: Remove trailing "..." from progress messages
2022-06-04 19:16:00 +02:00
MichaelEischer
60ca6b1418
Merge pull request #3774 from greatroar/archiver-pool
...
archiver: Remove cleanup goroutine from BufferPool
2022-06-04 18:50:24 +02:00
Jerome Küttner
9adaa6e240
Delete existing path before restoring a symlink
2022-06-01 17:26:25 +02:00
greatroar
b7c990871f
cmd/restic: Remove trailing "..." from progress messages
...
These were added after message since the last refactor of the progress
printing code. Also skips an allocation in the common case.
2022-05-31 19:06:26 +02:00
greatroar
0db1d11b2e
archiver: Remove cleanup goroutine from BufferPool
...
This isn't doing anything. Channels should get cleaned up by the GC when
the last reference to them disappears, just like all other data
structures. Also inlined BufferPool.Put in Buffer.Release, its only
caller.
2022-05-29 17:09:16 +02:00
Alexander Neumann
74f7fe2b98
Merge pull request #3767 from MichaelEischer/fix-prune-empty-snapshot
...
prune: Fix crash on snapshot loading error
2022-05-29 16:52:45 +02:00
Alexander Neumann
d2c5843c68
Merge pull request #3704 from MichaelEischer/compression-migrations
...
Support migration to repository format with compression
2022-05-29 15:52:21 +02:00
Alexander Neumann
78a21bbccf
Merge pull request #3752 from MichaelEischer/fix-dir-sync-errors
...
local: Ignore additional errors for directory syncing
2022-05-29 12:54:51 +02:00
MichaelEischer
2ce8587598
Merge pull request #3771 from greatroar/id-marshaljson
...
internal/restic: Custom ID.MarshalJSON
2022-05-28 16:24:51 +02:00
Michael Eischer
a73fc31b50
Fix linter check
2022-05-28 16:13:46 +02:00
greatroar
dde8e9e296
internal/restic: Custom ID.MarshalJSON
...
This skips an allocation. internal/archiver benchmarks, Linux/amd64:
name old time/op new time/op delta
ArchiverSaveFileSmall-8 3.94ms ± 6% 3.91ms ± 6% ~ (p=0.947 n=20+20)
ArchiverSaveFileLarge-8 304ms ± 3% 301ms ± 4% ~ (p=0.265 n=18+18)
name old speed new speed delta
ArchiverSaveFileSmall-8 1.04MB/s ± 6% 1.05MB/s ± 6% ~ (p=0.803 n=20+20)
ArchiverSaveFileLarge-8 142MB/s ± 3% 143MB/s ± 4% ~ (p=0.421 n=18+19)
name old alloc/op new alloc/op delta
ArchiverSaveFileSmall-8 17.9MB ± 0% 17.9MB ± 0% -0.01% (p=0.000 n=19+19)
ArchiverSaveFileLarge-8 382MB ± 2% 382MB ± 1% ~ (p=0.687 n=20+19)
name old allocs/op new allocs/op delta
ArchiverSaveFileSmall-8 540 ± 1% 528 ± 0% -2.19% (p=0.000 n=19+19)
ArchiverSaveFileLarge-8 1.93k ± 3% 1.79k ± 4% -7.06% (p=0.000 n=20+20)
2022-05-27 12:26:37 +02:00
Alexander Neumann
bc27c370e7
Update gopkg.in/yaml
...
This fixes a panic in invalid input, but I think we aren't affected.
2022-05-26 14:23:49 +02:00
Alexander Neumann
9e30152f3c
Merge pull request #3770 from lbausch/update-minio-go
...
Update github.com/minio/minio-go/v7 to v7.0.27
2022-05-26 14:21:19 +02:00
Lorenz Bausch
cc3f8d3732
Update github.com/minio/minio-go/v7 to v7.0.27
...
This version adds support for Cloudflare R2, as discussed in #3757
2022-05-26 13:05:13 +02:00
Michael Eischer
c8e1ac4049
prune: Don't print stack trace if snapshot can't be loaded
2022-05-23 22:38:45 +02:00
Michael Eischer
173695104c
prune: Fix crash on empty snapshot
2022-05-23 22:32:59 +02:00
MichaelEischer
ded783dd61
Merge pull request #3762 from marigbede/marigbede-gs-documentation-fix
...
Update 030_preparing_a_new_repo.rst
2022-05-23 22:10:38 +02:00
Arigbede Moses
2aa75d6272
Update 030_preparing_a_new_repo.rst
2022-05-18 22:03:59 +01:00
MichaelEischer
88a8701fb5
Merge pull request #3734 from lbausch/validate-patterns
...
Validate exclude patterns
2022-05-14 16:20:15 +02:00
MichaelEischer
b2a2e5f727
Merge pull request #3753 from greatroar/indexmap-alloc
...
repository: Re-tune indexmap allocation strategy
2022-05-14 15:44:08 +02:00
MichaelEischer
b52c631bd3
Merge pull request #3754 from greatroar/simplify-hashing
...
hashing: Fix up comments
2022-05-14 15:33:51 +02:00
Lorenz Bausch
e7df66cc91
Add changelog entry for validating exclude patterns
2022-05-11 22:41:05 +02:00
Lorenz Bausch
36bd464e8c
Add tests for validating exclude patterns
2022-05-11 22:41:00 +02:00
greatroar
39a335e690
hashing: Fix up comments
2022-05-11 21:36:10 +02:00
greatroar
5141228e0c
repository: Re-tune indexmap allocation strategy
...
fd05037e1a
changed the allocation batch
size from 256 to 128 under the assumption that an indexEntry is 60 bytes
on amd64, but it's 64: structs are padded out to a multiple of 8 for
alignment reasons. That means we'd waste no space in malloc even without
the batch allocation, at least on 64-bit machines. While that strategy
cuts the overallocation down dramatically for many small indexes, it also
seems to slow allocation down (Go 1.18, Linux, amd64, -benchtime=2s):
name old time/op new time/op delta
DecodeIndex-8 4.67s ± 5% 4.60s ± 1% ~ (p=0.953 n=10+5)
DecodeIndexParallel-8 4.67s ± 3% 4.60s ± 1% ~ (p=0.953 n=10+5)
IndexHasUnknown-8 37.8ns ± 8% 36.5ns ±14% ~ (p=0.841 n=5+5)
IndexHasKnown-8 38.5ns ±12% 37.7ns ±10% ~ (p=0.968 n=5+5)
IndexAlloc-8 615ms ±18% 607ms ± 1% ~ (p=1.000 n=10+5)
IndexAllocParallel-8 245ms ±11% 285ms ± 6% +16.40% (p=0.001 n=10+5)
MasterIndexAlloc-8 286ms ± 9% 275ms ± 2% ~ (p=1.000 n=10+5)
LoadIndex/v1-8 27.0ms ± 4% 26.8ms ± 1% ~ (p=0.690 n=5+5)
LoadIndex/v2-8 22.4ms ± 1% 22.8ms ± 2% +1.48% (p=0.016 n=5+5)
name old alloc/op new alloc/op delta
IndexAlloc-8 446MB ± 0% 446MB ± 0% -0.00% (p=0.000 n=8+4)
IndexAllocParallel-8 446MB ± 0% 446MB ± 0% -0.00% (p=0.008 n=8+5)
MasterIndexAlloc-8 213MB ± 0% 159MB ± 0% -25.47% (p=0.000 n=10+5)
name old allocs/op new allocs/op delta
IndexAlloc-8 913k ± 0% 2632k ± 0% +188.19% (p=0.008 n=5+5)
IndexAllocParallel-8 913k ± 0% 2632k ± 0% +188.21% (p=0.008 n=5+5)
MasterIndexAlloc-8 318k ± 0% 1172k ± 0% +267.86% (p=0.008 n=5+5)
Instead, this patch sets a batch size of 4, which means no space is
wasted by malloc on 64-bit and very little on 32-bit. It still gets very
close to the savings from not allocating in batches, without requiring
special code for bits.UintSize==64. Benchmark results, again for
Linux/amd64:
name old time/op new time/op delta
DecodeIndex-8 4.67s ± 5% 4.83s ± 9% ~ (p=0.315 n=10+10)
DecodeIndexParallel-8 4.67s ± 3% 4.68s ± 4% ~ (p=0.315 n=10+10)
IndexHasUnknown-8 37.8ns ± 8% 44.5ns ±19% ~ (p=0.095 n=5+5)
IndexHasKnown-8 38.5ns ±12% 36.9ns ± 8% ~ (p=0.690 n=5+5)
IndexAlloc-8 615ms ±18% 628ms ±18% ~ (p=0.218 n=10+10)
IndexAllocParallel-8 245ms ±11% 262ms ± 9% +7.02% (p=0.043 n=10+10)
MasterIndexAlloc-8 286ms ± 9% 287ms ±13% ~ (p=1.000 n=10+10)
LoadIndex/v1-8 27.0ms ± 4% 26.8ms ± 0% ~ (p=1.000 n=5+5)
LoadIndex/v2-8 22.4ms ± 1% 22.5ms ± 0% ~ (p=0.056 n=5+5)
name old alloc/op new alloc/op delta
IndexAlloc-8 446MB ± 0% 446MB ± 0% ~ (p=1.000 n=8+10)
IndexAllocParallel-8 446MB ± 0% 446MB ± 0% -0.00% (p=0.000 n=8+8)
MasterIndexAlloc-8 213MB ± 0% 160MB ± 0% -25.02% (p=0.000 n=10+9)
name old allocs/op new allocs/op delta
IndexAlloc-8 913k ± 0% 1333k ± 0% +45.94% (p=0.000 n=8+10)
IndexAllocParallel-8 913k ± 0% 1333k ± 0% +45.94% (p=0.000 n=8+8)
MasterIndexAlloc-8 318k ± 0% 525k ± 0% +64.99% (p=0.000 n=10+10)
The allocation method indexmap.newEntry has also been rewritten in a
form that is a few instructions shorter.
2022-05-11 21:22:14 +02:00
Michael Eischer
48a0d83143
local: Ignore additional errors for directory syncing
...
Apparently SMB/CIFS on Linux/macOS returns somewhat random errnos when
trying to sync a windows share which does not support calling fsync for
a directory.
2022-05-11 20:37:59 +02:00
MichaelEischer
ac36fda155
Merge pull request #3749 from greatroar/simplify-hashing
...
hashing: Remove io.WriterTo implementation
2022-05-11 20:03:43 +02:00
MichaelEischer
df554e5f69
Merge pull request #3748 from greatroar/runworkers
...
repository: Remove RunWorkers, report ctx.Err()
2022-05-11 19:38:46 +02:00
greatroar
54b8337813
hashing: Remove io.WriterTo implementation
...
This functionality has gone unused since
4b3dc415ef
changed hashing.Reader's only
client to use ioutil.ReadAll on a bufio.Reader wrapping the hashing
Reader.
Reverts bcb852a8d0
.
2022-05-10 23:41:18 +02:00
greatroar
2e0f1f5113
repository: Remove RunWorkers, report ctx.Err()
...
This removes RunWorkers, which had become mere overhead by successive
refactors. It also ensures that each former user of that function
returns any context error that occurs, so failure to complete an
operation is always reported as an error.
2022-05-10 22:26:00 +02:00
MichaelEischer
47c56dea5c
Merge pull request #3746 from greatroar/cache-lstat
...
cache: Don't Lstat before creating CACHEDIR.TAG
2022-05-10 20:31:15 +02:00
Alexander Neumann
c270ab1e08
Merge pull request #3744 from MichaelEischer/fix-windows-temp-file
...
Fix error on temp file deletion on windows
2022-05-10 19:53:25 +02:00
greatroar
2da377c582
cache: Don't Lstat before creating the tag file
...
The tag file is opened with O_CREATE|O_EXCL and ErrExist is handled, so
we don't need to check for existence first.
2022-05-10 18:52:39 +02:00
Michael Eischer
ae7e51382a
Fix error on temp file deletion on windows
...
Apparently it can take a moment between closing a tempfile marked as
DELETE_ON_CLOSE and it actually being deleted. During that time the file
is inaccessible. Thus just skip deleting the temp file on windows.
2022-05-09 22:43:26 +02:00
Michael Eischer
5c6db534d4
Add compression migration support to changelog
2022-05-09 22:39:02 +02:00
Michael Eischer
c1bbbcd0dc
migrate: Allow migrations to request a check run
...
This is currently only used by upgrade_repo_v2.
2022-05-09 22:31:30 +02:00
Michael Eischer
59eb132dcd
check: Better differentiate between warnings and errors
2022-05-09 22:31:30 +02:00
Michael Eischer
5815f727ee
checker: convert error type to use pointer-receivers
2022-05-09 22:31:30 +02:00
Michael Eischer
4faff0debe
doc: Describe repository upgrade process
2022-05-09 22:31:30 +02:00
Michael Eischer
e36a40db10
upgrade_repo_v2: Use atomic replace for supported backends
2022-05-09 22:31:30 +02:00
Michael Eischer
7559d2f105
Document repository version and minimum restic version
2022-05-09 22:31:30 +02:00
Michael Eischer
381bd94c6c
prune: Add option to repack uncompressed data
2022-05-09 22:31:30 +02:00
Michael Eischer
5406743102
prune: Automatically repack uncompressed trees for repo v2
...
Tree packs are cached locally at clients and thus benefit a lot from
being compressed. Ensure this be having prune always repack pack files
containing uncompressed trees.
2022-05-09 22:31:30 +02:00
Alexander Neumann
c8c0d659ec
Add migration to compress all data
2022-05-09 22:31:30 +02:00
Alexander Neumann
8c244214bf
Add tests for upgrade migration
2022-05-09 22:31:30 +02:00
Alexander Neumann
a5f1d318ac
Try to make repo upgrade migration more failsafe
2022-05-09 22:31:30 +02:00
Alexander Neumann
82ed5a3a15
Add repo upgrade migration
2022-05-09 22:31:30 +02:00
Alexander Neumann
3af6c180e4
Improve migrate command
2022-05-09 22:31:30 +02:00
Michael Eischer
92816fa966
init: Enable compression support by default
2022-05-09 22:31:30 +02:00
MichaelEischer
ab49c14621
Merge pull request #3740 from MichaelEischer/fix-restore-size-stats
...
Fix restore size calculation for multiple snapshots
2022-05-09 21:39:45 +02:00
Michael Eischer
2c07f7fff3
stats: hardlinks only reduce restore within a snapshot
...
The `stats` command checks inodes to not count hardlinked files multiple
times into the restore size. This check applies across all snapshots and
not only within snapshots. As a result the result size was far too low
when calculating it for multiple snapshots and it would vary depending
on the order in which snapshots were listed.
2022-05-09 21:26:24 +02:00
Lorenz Bausch
9fb81c4246
Validate exclude patterns
2022-05-07 21:12:47 +02:00
Lorenz Bausch
e7fd200237
Keep original pattern for later use
2022-05-07 21:08:09 +02:00
MichaelEischer
cc8a03b1d0
Merge pull request #3735 from HenrikBengtsson/master
...
DOCS: Incorrectly used SI units when IEC units were meant (fix #3669 )
2022-05-07 11:15:27 +02:00
Henrik Bengtsson
9bb532672a
DOCS: Incorrectly used SI units when IEC units were meant ( fix #3669 )
2022-05-02 21:21:39 -07:00
MichaelEischer
26c333325c
Merge pull request #3715 from ema/master
...
doc: specify AWS Region via AWS_DEFAULT_REGION
2022-05-02 21:46:56 +02:00
Emanuele Rocca
38c0531b52
doc: specify AWS Region via AWS_DEFAULT_REGION
...
If no specific Region is mentioned in RESTIC_REPOSITORY, AWS defaults to
us-east-1. For this reason, users that follow the tutorial and create
their S3 bucket in any other region get the following error:
"Fatal: create repository at [...] client.BucketExists"
Explicitly specifying the AWS region name fixes the issue.
2022-05-02 21:26:58 +02:00
Alexander Neumann
fb5b9370f3
Merge pull request #3728 from MichaelEischer/debug-examine-upload
...
Add support to reupload blobs in `debug examine`
2022-04-30 20:39:19 +02:00
Alexander Neumann
ffbd48c0c6
Merge pull request #3481 from MichaelEischer/recover-enospace
...
Recover from no free space errors
2022-04-30 20:22:26 +02:00
Michael Eischer
95bcc9ea31
debug: Support pack ID prefixes in `debug examine`
2022-04-30 20:20:31 +02:00
Michael Eischer
2d6a943911
debug: Add switch to upload blobs extracted by debug examine
...
This simplifies salvaging a damaged pack file. Reuploading tree blobs
was previously not possible.
2022-04-30 20:20:31 +02:00
Alexander Neumann
9af499d8a4
Tidy go.sum
2022-04-30 20:16:09 +02:00
Alexander Neumann
2e3d23c1d7
Update github.com/klauspost/compress
2022-04-30 20:03:21 +02:00
Michael Eischer
dbbeac7174
prune: Add unsafe option to recover from no free space
...
The new option allows prune to operate with nearly no scratch space by only removing
no longer necessary pack files and first deleting the index before
rebuilding it. By first deleting the index it becomes safe to just
delete no longer necessary pack files. However, as a downside there's
now the risk that the repository becomes inaccessible if prune fails.
To recover from that problem a user might have to manually delete the
repository index and then run (a full) `rebuild-index` again.
2022-04-30 19:21:07 +02:00
Michael Eischer
cf5cb673fb
repository: Use existing method to collect pack ids
2022-04-30 19:14:21 +02:00
Michael Eischer
b335cb6285
repository: Refactor index IDs collection
2022-04-30 19:14:21 +02:00
MichaelEischer
9c047f170a
Merge pull request #3419 from DanielG/relax-file-modes
...
Enable admin to decide group access to repository files
2022-04-30 16:39:37 +02:00
Daniel Gröber
f31b4f29c1
Use config file modes to derive new dir/file modes
...
Fixes #2351
2022-04-30 15:59:51 +02:00
Alexander Neumann
71c653f9e0
Merge pull request #3727 from MichaelEischer/changelog-3475
...
Add changelog for local/sftp connection limit
2022-04-30 14:05:21 +02:00
Michael Eischer
29a8f92967
Add changelog for local/sftp connection limit
2022-04-30 13:36:47 +02:00
MichaelEischer
ac9324aeaf
Merge pull request #3666 from MichaelEischer/compression
...
Implement compression support
2022-04-30 11:49:05 +02:00
Alexander Neumann
dc5adef255
Add documentation for --repository-version
2022-04-30 11:34:10 +02:00
Michael Eischer
4b01b06f2f
repository: Test compressed blobs in StreamPack
2022-04-30 11:34:10 +02:00
Michael Eischer
bcab548617
pack: slightly expand testing of compressed blobs
2022-04-30 11:34:10 +02:00
Michael Eischer
ec2b25565a
repository: test uncompressedLength field and index example
2022-04-30 11:34:10 +02:00
Michael Eischer
9ffb8920f1
repository: run blackbox tests using old and new repo version
2022-04-30 11:34:10 +02:00
Michael Eischer
abe5935693
repository: unify repository version-specific initialization
...
Mark the master index as compressed also when initializing a new
repository. This is only relevant for testing.
2022-04-30 11:34:10 +02:00
Alexander Neumann
8776031f96
Leave allocating slices to the decompress code
2022-04-30 11:34:10 +02:00
Alexander Neumann
5eb05a0afe
Configure zstd encoder/decoder
2022-04-30 11:34:10 +02:00
Michael Eischer
2f36e044db
Cleanup pack header check
2022-04-30 11:34:10 +02:00
Alexander Neumann
94dc9a0fa7
Amend changelog
2022-04-30 11:34:10 +02:00
Alexander Neumann
8b11b86383
Add option global --compression
2022-04-30 11:34:10 +02:00
Michael Eischer
f38f457a64
Add basic changelog for compression support
2022-04-30 11:34:10 +02:00
Michael Eischer
ba27d29d58
Print repository version when opening a repo
2022-04-30 11:34:10 +02:00
Michael Eischer
7132df529e
repository: Increase index size for repo version 2
...
A compressed index is only about one third the size of an uncompressed
one. Thus increase the number of entries in an index to avoid cluttering
the repository with small indexes.
2022-04-30 11:34:10 +02:00
Michael Eischer
2535524132
debug: Add support for compressed blobs
2022-04-30 11:34:10 +02:00
Michael Eischer
fda7bb0f09
debug: Reduce code duplication
2022-04-30 11:34:10 +02:00
Michael Eischer
66f9048bce
repository: Alloc zstd encoder/decoder on demand
2022-04-30 11:34:10 +02:00
Michael Eischer
fd05037e1a
repository: recalibrate index batch allocation size
2022-04-30 11:34:10 +02:00
Michael Eischer
6fb408d90e
repository: implement pack compression
2022-04-30 11:34:10 +02:00
Michael Eischer
362ab06023
init: Add flag to specify created repository version
2022-04-30 10:07:42 +02:00
Michael Eischer
4b957e7373
repository: Implement index/snapshot/lock compression
...
The config file is not compressed as it should remain readable by older
restic versions such that these can return a proper error.
As the old format for unpacked data does not include a version header,
make use of a trick: The old data is always encoded as JSON. Thus it can
only start with '{' or '['. For any other value the first byte indicates
a versioned format. The version is set to 2 for now. Then the zstd
compressed data follows.
2022-04-30 10:07:42 +02:00
Michael Eischer
0957b74887
Misc design.rst cleanups
2022-04-30 10:07:42 +02:00
Alexander Neumann
270ed00d1f
doc: Add repository compression support documentation
...
Co-authored-by: Michael Eischer <michael.eischer@fau.de>
2022-04-30 10:07:42 +02:00
Alexander Neumann
4e1ef7804a
Merge pull request #3717 from MichaelEischer/fix-stuck-repack
...
Fix stuck repack step
2022-04-30 09:50:43 +02:00
Alexander Neumann
e4780d3956
Merge pull request #3718 from MichaelEischer/sftp-docs-fix
...
doc: sftp with password actually works
2022-04-23 20:27:27 +02:00
Alexander Neumann
c183e35b5a
Merge pull request #3719 from MichaelEischer/read-write-order
...
Describe repository read/write order required by repository format
2022-04-23 15:23:00 +02:00
Michael Eischer
6f9e20a1bb
doc: Describe repository read/write order
2022-04-23 15:21:02 +02:00
Michael Eischer
f9219e8608
doc: sftp with password actually works
2022-04-23 11:57:36 +02:00
Michael Eischer
3b630d9998
add missing streamPacks changelog
2022-04-23 11:50:19 +02:00
Michael Eischer
566ac11c65
fix changelog name
2022-04-23 11:37:00 +02:00
Michael Eischer
f5609d1d3c
prune: Fail early if too few backend connections
2022-04-23 11:32:52 +02:00
Michael Eischer
e597b99b55
repository: Reduce repack workers to prevent deadlock
...
As repack streams packs these occupy one backend connection. Uploading a
new pack also requires a backend connection. To prevent a deadlock
during repack when reaching the backend connections limit, simply limit
the repackWorker count to always leave one connection for uploading.
2022-04-23 11:28:18 +02:00
Michael Eischer
ee627cd832
backend/mem: Actually enforce connection limit
...
This will allow tests to detect deadlocks related to the connections
limit.
2022-04-23 11:22:00 +02:00
Michael Eischer
4f97492d28
Backend: Expose connections parameter
2022-04-23 11:13:08 +02:00
rawtaz
07a565e6f7
Merge pull request #3716 from MichaelEischer/password-error-on-stderr
...
Print password error message on stderr
2022-04-21 01:40:54 +02:00
Michael Eischer
bf7da7ff10
Print password error message on stderr
...
The password prompt itself is already printed on stderr.
2022-04-20 22:22:09 +02:00
Alexander Neumann
dba47d29d5
Merge pull request #3711 from restic/doc-grouping
...
doc: Clarify and make grouping in forget more noticeable
2022-04-16 10:08:11 +02:00
Leo R. Lundgren
8ac7519fd5
doc: Clarify and make grouping in forget more noticeable
2022-04-16 01:13:13 +02:00
Alexander Neumann
edc1a24a90
Merge pull request #3707 from duracell/patch-1
...
doc: fix missing "init" in rest-server example
2022-04-12 20:39:25 +02:00
Michael
9563e2f75c
doc: fix missing "init" in rest-server example
2022-04-12 18:21:53 +02:00
Alexander Neumann
7f133a28b2
Update VERSION file
2022-04-11 20:34:14 +02:00
Alexander Neumann
4f3b1f19cb
Set development version for 0.13.1
2022-04-10 20:41:00 +02:00
Alexander Neumann
89ee1cf9ee
Merge pull request #3610 from MichaelEischer/windows-temp-files
...
Improve handling of temporary files on windows
2022-04-10 20:29:04 +02:00
Alexander Neumann
a059ef90f8
Merge pull request #3702 from MichaelEischer/extend-config-error
...
Print used key name if config fails to load
2022-04-10 20:25:24 +02:00
Michael Eischer
4077a81b34
Add simple test for fs.TempFile on windows
2022-04-09 23:37:58 +02:00
Michael Eischer
9a3f1a9703
Simplify and comment TempFile implementation for windows
2022-04-09 23:37:58 +02:00
Michael Eischer
c2aabb2686
Print used key name if config fails to load
2022-04-09 22:38:18 +02:00
MichaelEischer
c60a5f00c9
Merge pull request #3675 from ItsMattL/update
...
Refactor file handing for self-update.
2022-04-09 21:55:56 +02:00
Matt LaPlante
0ba9d4ced7
Refactor file handing for self-update.
...
* Write new file payload to a temp file before touching the original
binary. Minimizes the possibility of failing mid-write and corrupting
the binary.
* On Windows, move the original binary out to a temp file rather than
removing it as the running binary is locked. Fixes issue #2248 .
2022-04-09 21:40:33 +02:00
Alexander Neumann
04e054465a
Merge pull request #3475 from MichaelEischer/local-sftp-conn-limit
...
Limit concurrent operations for local / sftp backend
2022-04-09 21:33:00 +02:00
Alexander Neumann
1519e9f911
Merge pull request #3570 from MichaelEischer/list-snapshots-before-index
...
List snapshots before index
2022-04-09 21:15:49 +02:00
Michael Eischer
ebab35581c
Check in integration test that snapshots are listed before the index
...
As an exception prune is still allowed to load the index before
snapshots, as it uses exclusive locks. In case of problems with locking
it is also better to load snapshots created after loading the index, as
this will lead to a prune sanity check failure instead of a broken snapshot.
2022-04-09 12:27:27 +02:00
Michael Eischer
7b9ae91e04
copy: Load snapshots before indexes
2022-04-09 12:27:25 +02:00
Michael Eischer
47243176fa
diff: list snapshots only once
2022-04-09 12:26:31 +02:00
Michael Eischer
5af828e3e6
add changelogs
2022-04-09 12:26:31 +02:00
Michael Eischer
4636c20397
test that TestFindListOnce calls List only once
2022-04-09 12:26:31 +02:00
Michael Eischer
9e12159230
Fix O(n) backend list calls in FindFilteredSnapshots
...
When resolving snapshotIDs in FindFilteredSnapshots either
FindLatestSnapshot or FindSnapshot is called. Both operations issue a
list operation to the backend. When for example passing a long list of
snapshot ids to `forget` this could lead to a large number of list
operations.
2022-04-09 12:26:31 +02:00
Michael Eischer
3d29083e60
copy/find/ls/recover/stats: Memorize snapshot listing before index
...
These commands filter the snapshots according to some criteria which
essentially requires loading the index before filtering the snapshots.
Thus create a copy of the snapshots list beforehand and use it later on.
2022-04-09 12:26:30 +02:00
Michael Eischer
2ec0f3303a
backup/diff/dump/restore/stats: List snapshots before index
...
During a backup the index is written before the corresponding snapshots.
To ensure that a concurrent/later restic run can read a snapshot's data,
restic thus must first load the snapshots and only afterwards the index.
Otherwise it is not possible to ensure that the loaded index is recent
enough to cover all of the snapshot's data.
2022-04-09 12:24:09 +02:00
Michael Eischer
ece06f125e
sftp: Limit concurrent backend operations
2022-04-09 12:21:38 +02:00
Michael Eischer
cd783358d3
local: Limit concurrent backend operations
...
Use a limit of 2 similar to the filereader concurrency in the archiver.
2022-04-09 12:21:38 +02:00
Michael Eischer
0b258cc054
backends: clean reader closing
2022-04-09 12:21:38 +02:00
Alex Duchesne
9e34c791c9
Better temp file cleanup on Windows.
2022-04-09 12:00:22 +02:00
Alexander Neumann
7d55b4f95e
Merge pull request #3701 from restic/rawtaz-doc-exclude-cachedir
...
doc: Link to CACHEDIR.TAG specification
2022-04-08 09:35:08 +02:00
rawtaz
de4e3117eb
doc: Link to CACHEDIR.TAG specification
2022-04-08 03:07:27 +02:00
MichaelEischer
500079d265
Merge pull request #3689 from brightdroid/patch-1
...
added documentation for zsh autocompletion
2022-04-03 21:49:47 +02:00
Alexander Neumann
192288bc9c
Merge pull request #3696 from cqjjjzr/fix-win-procgrp
...
Fix rclone (scoop shim) and sftp issue due to detached console on Windows
2022-04-03 13:46:28 +02:00
Charlie Jiang
d9c9415cfd
Fix rclone (scoop shim) and sftp issue due to detached console on Windows
2022-04-03 17:53:17 +08:00
Alexander Neumann
59370b6062
Merge pull request #3695 from jernej-9/typo-fix
...
Fix a typo in the docs
2022-04-01 20:10:40 +02:00
Jernej Debevc
6e5731bf2f
Fix a typo in the docs
2022-04-01 17:16:55 +02:00
Alexander Neumann
305cd1e730
Merge pull request #3693 from greatroar/cast-btrfs-super-magic
...
Cast unix.Statfs_t.Type to int64 when checking for btrfs
2022-04-01 08:09:46 +02:00
greatroar
c23c0f7c14
Cast unix.Statfs_t.Type to int64 when checking for btrfs
...
Fixes #3687 . Uses the cast suggested by @MichaelEischer, except that the
contant isn't cast along, because it's untyped and will be converted by
the compiler as necessary.
2022-03-31 22:30:45 +02:00
Alexander Neumann
774c2e75ca
Merge pull request #3680 from restic/update-deps
...
Update all dependencies (except fuse), require Go 1.15
2022-03-30 21:36:08 +02:00
Alexander Neumann
66d50b72e3
Require Go 1.15 or later
...
The library github.com/golang-jwt/jwt/v4 requires the FillByte() method
of *big.Int, so we're raising the minimum Go version to 1.15.
2022-03-30 21:11:17 +02:00
Alexander Neumann
89d86a7933
Update all dependencies (except fuse)
2022-03-30 21:11:13 +02:00
Christoph Roeder
7510bdc247
added documentation for zsh autocompletion
2022-03-30 10:45:52 +02:00
Alexander Neumann
f190d2e60e
Merge pull request #3686 from restic/fix-diff
...
Fix diff
2022-03-30 07:43:07 +02:00
Alexander Neumann
206550a246
Fix diff
...
Nodes in trees were always printed with a `+` in diff, regardless of
whether or not a dir was added or removed. Let's use the mode we were
passed in printDir().
Closes #3685
2022-03-29 21:05:11 +02:00
Alexander Neumann
db8a958991
Merge pull request #3683 from MichaelEischer/fix-golangci-lint-warnings
...
Fix golangci lint warnings
2022-03-29 11:45:10 +02:00
Alexander Neumann
724ace0e99
Merge pull request #3682 from MichaelEischer/refactor-code
...
Pack size calculation cleanup and misc other changes
2022-03-29 11:06:04 +02:00
Michael Eischer
af31266b7d
golangci-lint: replace deprecated golint with revive
2022-03-28 22:33:17 +02:00
Michael Eischer
2f81af6afa
bloblru: Fix comment for New function
2022-03-28 22:25:25 +02:00
Michael Eischer
61e179ee78
switch to golang.org/x/term
2022-03-28 22:24:15 +02:00
Michael Eischer
c60540b196
add go:build headers everywhere
2022-03-28 22:23:47 +02:00
Michael Eischer
fefe9f5c0e
pack: Hide more implementation details
2022-03-28 22:12:16 +02:00
Michael Eischer
a773cb6527
pack: cleanup header size calculation
2022-03-28 22:09:49 +02:00
Michael Eischer
6408686973
repository: Simplify Blob equality check
2022-03-28 22:09:49 +02:00
Michael Eischer
243698680a
crypto: Use helpers for size calculations
2022-03-28 22:09:49 +02:00
Michael Eischer
d6db5a1fc2
archiver: Fix test
...
The test relied on an undeocumented sideeffect of the LoadBlob implementation
2022-03-28 22:09:49 +02:00
Michael Eischer
f78bd14e28
repository: Remove pack implementation details from MasterIndex
2022-03-28 22:09:49 +02:00
Michael Eischer
dc3d77dacc
repository: make saveAndEncrypt private
2022-03-28 22:09:49 +02:00
Michael Eischer
6877e7edbb
repository: Rename LoadAndDecrypt to LoadUnpacked
...
The method is the complement for SaveUnpacked and not for
SaveAndEncrypt. The latter assembles blobs into pack files.
2022-03-28 22:09:49 +02:00
Michael Eischer
2e1613d4c6
errors: Ensure that errors.IsFatal(errors.Fatal("err")) == true
...
This fixes a few cases where restic output "Fatal: Fatal: [...]"
2022-03-28 22:09:49 +02:00
Alexander Neumann
a08b95c497
Merge pull request #3513 from MichaelEischer/fast-copy
...
Speed-up copy command
2022-03-28 20:18:43 +02:00
Michael Eischer
537b4c310a
copy: Implement by reusing repack
...
The repack operation copies all selected blobs from a set of pack files
into new pack files. For prune the source and destination repositories
are identical. To implement copy, just use a different source and
destination repository.
2022-03-26 20:47:15 +01:00
Alexander Neumann
4d5db61bd0
Merge pull request #3484 from MichaelEischer/stream-check-repack
...
Stream packs in `check --read-data` and during repacking
2022-03-26 20:46:17 +01:00
Alexander Neumann
4ab12f59a5
Set development version for 0.13.0
2022-03-26 20:10:07 +01:00
Alexander Neumann
e682f7c0d6
Add tests for StreamPack
2022-03-21 21:15:03 +01:00
Michael Eischer
27524979e8
restorer: Remove dead code
2022-02-13 11:43:09 +01:00
Michael Eischer
bba8ba7a5b
repository: cancel streampack context after error
2022-02-12 20:18:25 +01:00
Michael Eischer
47554a3428
repository: Fix error handling in repack
...
When storing a blob fails, this is a fatal error which must not be
retried.
2022-02-12 20:18:25 +01:00
Michael Eischer
4b3dc415ef
checker: cleanup header extraction
2022-02-12 20:18:25 +01:00
Michael Eischer
930a00ad54
checker: reuse bufio reader
2022-02-12 20:18:25 +01:00
Michael Eischer
34ebafb8b6
repository: don't crash if blob size is too short
2022-02-12 20:18:25 +01:00
Michael Eischer
becebf5d88
repository: remove unused DownloadAndHash
2022-02-12 20:18:25 +01:00
Michael Eischer
f1e58e7c7f
checker: rewrite ReadData to stream packs
2022-02-12 20:18:25 +01:00
Michael Eischer
f40abd92fa
restorer: convert to use StreamPack
2022-02-12 20:18:25 +01:00
Michael Eischer
f00f690658
repository: stream packs during repacking
2022-02-12 20:18:25 +01:00
Michael Eischer
c4a2bfcb39
repository: Add StreamPacks function
...
The function supports efficiently loading a specified list of blobs from
a single pack in a streaming fashion. That is there's no need for
temporary files independent of the pack size.
2022-02-12 20:18:25 +01:00
Michael Eischer
153e2ba859
repository: Implement lisiting blobs per pack file
2022-02-12 20:18:24 +01:00