37a5e2d681
rest: use global context on repository creation
2020-10-09 22:39:06 +02:00
4b0fcaed45
unlock: use proper context for locks cleanup
...
The list operation used by RemoveStaleLocks or RemoveAllLocks will
already be canceled by the passed in context. Therefore we can also just
cancel the remove operation as the unlock command won't process all lock
files anyways.
2020-10-09 22:37:56 +02:00
645a6efaf2
restorer: remove redundant type specification
2020-10-09 22:37:56 +02:00
dc2e664209
integration_fuse_test: use global context
...
No need to use the TODO context.
2020-10-09 22:37:56 +02:00
a449450021
init: pass proper context to master key generation
...
This is no change in behavior as a canceled context did later on cause
the config file creation to fail. Therefore this change just lets the
repository initialization fail a bit earlier.
2020-10-09 22:37:56 +02:00
603bb0e309
restore: Use proper context while loading snapshot
2020-10-09 22:37:56 +02:00
27456f6545
debug: use proper context
...
This allows the debug commands to be properly interrupted.
2020-10-09 22:37:56 +02:00
c458e114d4
pass context to Find / FindSnapshot
...
This allows proper interruption of restic while it searches for
snapshots or key files.
2020-10-09 22:37:56 +02:00
45e9a55c62
Wire context into backend layout detection
2020-10-09 22:37:24 +02:00
307a6ba3a3
Upgrade minio sdk to v7
...
This changes are primarily straightforward modifications to pass the
parameters in the now expected way.
2020-10-09 22:37:24 +02:00
50da20d93d
Warn if backup failed to read tree blob
2020-10-09 22:36:27 +02:00
5fd3dbccb7
Merge pull request #2963 from MichaelEischer/fix-status-deadlock
...
backup: Fix possible deadlock of scanner goroutine
2020-10-09 21:35:50 +02:00
35655a481b
Merge pull request #2931 from MichaelEischer/reduce-debug-overhead
...
Reduce debug log overhead
2020-10-09 21:35:11 +02:00
30cb553c8d
Merge pull request #2932 from MichaelEischer/proper-rclone-create
...
Call rclone.Create to create a new repository for the rclone backend
2020-10-09 21:29:15 +02:00
2f3eeff2e7
Merge pull request #2964 from MichaelEischer/fix-repro-instructions
...
Fix instructions to reproduce the release binaries
2020-10-09 21:28:24 +02:00
356ac404cd
Merge pull request #2996 from greatroar/setconsolecursorposition
...
Use windows.SetConsoleCursorPosition in ui/termstatus
2020-10-09 18:19:02 +02:00
ab769abeaf
Merge pull request #3002 from MichaelEischer/update-contributing
...
Update links to help wanted labels in CONTRIBUTING.md and link to forum
2020-10-09 17:24:25 +02:00
4036991c91
Update links to help wanted labels in CONTRIBUTING.md and link to forum
2020-10-09 16:15:43 +02:00
88c8e903d2
filter: Fix glob matching on absolute path marker on windows
...
A pattern part containing "/" is used to mark a path or a pattern as
absolute. However, on Windows the path separator is "\" such that glob
patterns like "?" could match the marker. The code now explicitly skips
the marker when the pattern does not represent an absolute path.
2020-10-09 16:11:05 +02:00
740758a5fa
Optimize filter pattern matching
...
By replacing "**" with "", checking for this special path component can
be reduced to a length-zero check.
name old time/op new time/op delta
FilterLines-8 44.7ms ± 5% 44.9ms ± 5% ~ (p=0.631 n=10+10)
FilterPatterns/Relative-8 13.6ms ± 4% 13.4ms ± 5% ~ (p=0.165 n=10+10)
FilterPatterns/Absolute-8 10.9ms ± 5% 10.7ms ± 4% ~ (p=0.052 n=10+10)
FilterPatterns/Wildcard-8 53.7ms ± 5% 50.4ms ± 5% -6.00% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-8 128ms ± 2% 95ms ± 1% -25.54% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-8 3.57MB ± 0% 3.57MB ± 0% ~ (p=1.000 n=9+8)
FilterPatterns/Absolute-8 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.903 n=9+8)
FilterPatterns/Wildcard-8 19.7MB ± 0% 19.7MB ± 0% -0.00% (p=0.022 n=10+9)
FilterPatterns/ManyNoMatch-8 3.57MB ± 0% 3.57MB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-8 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-8 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-8 88.7k ± 0% 88.7k ± 0% ~ (all equal)
FilterPatterns/ManyNoMatch-8 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-09 15:51:14 +02:00
862ee4b2c9
Merge pull request #2970 from labkode/fopenskip
...
Skip fopen for file change check to avoid network penalty
2020-10-09 11:43:54 +02:00
958dc6aafc
Skip fopen for file change check to avoid network penalty
2020-10-09 10:53:28 +02:00
88cc444779
Merge pull request #2934 from MichaelEischer/upgrade-backoff
...
Upgrade github.com/cenkalti/backoff module
2020-10-08 20:33:30 +02:00
4a424af1d5
Merge pull request #2989 from MichaelEischer/remove-local-chmod
...
local: mark repository files as read-only and handle chmod errors
2020-10-08 19:04:13 +02:00
bc65da2baf
Add changelog
2020-10-08 14:45:26 +02:00
b79f18209f
Upgrade github.com/cenkalti/backoff module
...
We now use v4 of the module. `backoff.WithMaxRetries` no longer repeats
an operation endlessly when a retry count of 0 is specified. This
required a few fixes for the tests.
2020-10-07 22:04:59 +02:00
8388e67c4b
filter: cleanup path separator conversion
2020-10-07 21:14:07 +02:00
0acc3c5923
filter: special case patterns without globbing characters
...
In case a part of a path is a simple string, we can just check for
equality without complex parsing in filepath.Match.
name old time/op new time/op delta
FilterLines-4 34.8ms ±17% 41.2ms ±23% +18.36% (p=0.000 n=10+10)
FilterPatterns/Relative-4 21.7ms ± 6% 12.1ms ±23% -44.46% (p=0.000 n=10+10)
FilterPatterns/Absolute-4 10.0ms ± 5% 9.1ms ±11% -9.80% (p=0.006 n=10+9)
FilterPatterns/Wildcard-4 47.0ms ± 7% 42.2ms ± 5% -10.19% (p=0.000 n=9+10)
FilterPatterns/ManyNoMatch-4 190ms ± 1% 131ms ±20% -31.47% (p=0.000 n=8+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.870 n=9+9)
FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.145 n=10+10)
FilterPatterns/Wildcard-4 14.3MB ± 0% 19.7MB ± 0% +37.91% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.421 n=10+9)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-4 88.7k ± 0% 88.7k ± 0% ~ (all equal)
FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-07 20:55:43 +02:00
54a124de3b
filter: explicitly test separate ListWithChild function
2020-10-07 20:47:52 +02:00
bcc3bddcf4
filter: only check whether a child path could match when necessary
...
When checking excludes there is no need to test whether a child path
could also match the pattern, as it is by definition excluded.
Previously childMayMatch was calculated but then discarded. For simple
absolute paths this can account for half the time spent for checking
pattern matches.
name old time/op new time/op delta
FilterPatterns/Relative-4 23.3ms ± 9% 21.7ms ± 6% -6.68% (p=0.004 n=10+10)
FilterPatterns/Absolute-4 13.9ms ± 7% 10.0ms ± 5% -27.61% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 51.4ms ± 7% 47.0ms ± 7% -8.51% (p=0.001 n=9+9)
FilterPatterns/ManyNoMatch-4 551ms ± 9% 190ms ± 1% -65.41% (p=0.000 n=10+8)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.665 n=10+9)
FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.480 n=9+10)
FilterPatterns/Wildcard-4 14.3MB ± 0% 14.3MB ± 0% ~ (p=0.431 n=9+10)
FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-4 88.7k ± 0% 88.7k ± 0% ~ (all equal)
FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-07 20:47:52 +02:00
17c53efb0d
filter: Optimize double wildcard expansion
...
This only allocates a single slice to expand the double wildcard and
only copies the pattern prefix once.
name old time/op new time/op delta
FilterPatterns/Relative-4 22.7ms ± 5% 23.3ms ± 9% ~ (p=0.353 n=10+10)
FilterPatterns/Absolute-4 14.2ms ±13% 13.9ms ± 7% ~ (p=0.853 n=10+10)
FilterPatterns/Wildcard-4 266ms ±16% 51ms ± 7% -80.67% (p=0.000 n=10+9)
FilterPatterns/ManyNoMatch-4 554ms ± 6% 551ms ± 9% ~ (p=0.436 n=10+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.349 n=10+10)
FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.073 n=10+9)
FilterPatterns/Wildcard-4 141MB ± 0% 14MB ± 0% -89.89% (p=0.000 n=10+9)
FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-4 1.63M ± 0% 0.09M ± 0% -94.56% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-07 20:47:48 +02:00
7959796269
filter: Special case for absolute paths
...
name old time/op new time/op delta
FilterPatterns/Relative-4 23.6ms ±20% 22.7ms ± 5% ~ (p=0.684 n=10+10)
FilterPatterns/Absolute-4 32.3ms ± 8% 14.2ms ±13% -56.01% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 334ms ±17% 266ms ±16% -20.56% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 709ms ± 7% 554ms ± 6% -21.89% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% +0.00% (p=0.046 n=9+10)
FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.464 n=10+10)
FilterPatterns/Wildcard-4 141MB ± 0% 141MB ± 0% ~ (p=0.163 n=9+10)
FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-4 1.63M ± 0% 1.63M ± 0% ~ (p=0.072 n=10+10)
FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-07 20:47:29 +02:00
375c2a56de
filter: Parse filter patterns only once
...
name old time/op new time/op delta
FilterPatterns/Relative-4 30.3ms ±10% 23.6ms ±20% -22.12% (p=0.000 n=10+10)
FilterPatterns/Absolute-4 49.0ms ± 3% 32.3ms ± 8% -33.94% (p=0.000 n=8+10)
FilterPatterns/Wildcard-4 345ms ± 9% 334ms ±17% ~ (p=0.315 n=10+10)
FilterPatterns/ManyNoMatch-4 3.93s ± 2% 0.71s ± 7% -81.98% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 4.63MB ± 0% 3.57MB ± 0% -22.98% (p=0.000 n=9+9)
FilterPatterns/Absolute-4 8.54MB ± 0% 3.57MB ± 0% -58.20% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 146MB ± 0% 141MB ± 0% -2.93% (p=0.000 n=9+9)
FilterPatterns/ManyNoMatch-4 907MB ± 0% 4MB ± 0% -99.61% (p=0.000 n=9+9)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 66.6k ± 0% 22.2k ± 0% -66.67% (p=0.000 n=10+10)
FilterPatterns/Absolute-4 88.7k ± 0% 22.2k ± 0% -75.00% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 1.70M ± 0% 1.63M ± 0% -3.92% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 4.46M ± 0% 0.02M ± 0% -99.50% (p=0.000 n=10+10)
2020-10-07 20:47:27 +02:00
b8eacd1364
filter: Reduce redundant path and pattern splitting
...
A single call to filter.List will split the path only once and also
split each search pattern only once and use it for both match and
childMatch.
name old time/op new time/op delta
FilterPatterns/Relative-4 62.1ms ±15% 30.3ms ±10% -51.22% (p=0.000 n=9+10)
FilterPatterns/Absolute-4 111ms ±10% 49ms ± 3% -56.08% (p=0.000 n=10+8)
FilterPatterns/Wildcard-4 393ms ±15% 345ms ± 9% -12.30% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 10.0s ± 3% 3.9s ± 2% -60.53% (p=0.000 n=10+9)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 16.4MB ± 0% 4.6MB ± 0% -71.76% (p=0.000 n=10+9)
FilterPatterns/Absolute-4 31.4MB ± 0% 8.5MB ± 0% -72.77% (p=0.000 n=9+10)
FilterPatterns/Wildcard-4 168MB ± 0% 146MB ± 0% -13.19% (p=0.000 n=10+9)
FilterPatterns/ManyNoMatch-4 3.23GB ± 0% 0.91GB ± 0% -71.96% (p=0.000 n=10+9)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 178k ± 0% 67k ± 0% -62.50% (p=0.000 n=10+10)
FilterPatterns/Absolute-4 266k ± 0% 89k ± 0% -66.67% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 1.87M ± 0% 1.70M ± 0% -9.47% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 17.7M ± 0% 4.5M ± 0% -74.87% (p=0.000 n=9+10)
2020-10-07 18:13:19 +02:00
e73c281142
filter: Benchmark absolute paths, wildcards and long filter lists
2020-10-07 17:54:36 +02:00
fdd3b14db3
filter: test some corner cases
2020-10-07 17:09:44 +02:00
f4282aa6fd
local: mark repository files as read-only
...
This is intended to prevent accidental modifications of data files.
Marking the files as read-only was accidentally removed in #1258 .
2020-10-07 12:29:37 +02:00
40ee17167e
local: Ignore permission errors on chmod call in Save/Remove operation
...
The file is already created with the proper permissions, thus the chmod
call is not critical. However, some file systems have don't allow
modifications of the file permissions. Similarly the chmod call in the Remove
operation should not prevent it from working.
2020-10-07 12:29:37 +02:00
f3e933f0c1
Use windows.SetConsoleCursorPosition in ui/termstatus
2020-10-07 11:18:54 +02:00
0ae02f3030
Merge pull request #2844 from aawsome/prune-integration-tests
...
Add prune integration tests for many edge cases
2020-10-06 23:16:41 +02:00
e401859afb
Merge pull request #2992 from lbausch/repo_wording
...
Replace repo/repos with repository/repositories in `--no-lock` help text
2020-10-06 23:03:55 +02:00
af4100e07d
Replace repo/repos with repository/repositories in --no-lock help text
2020-10-06 22:49:48 +02:00
ea5bbe0857
Remove dysfunctional link to saythanks
2020-10-06 20:43:44 +02:00
6822a58413
Add prune integration tests for many edge cases
2020-10-06 20:20:05 +02:00
09d39e260d
Merge pull request #2980 from greatroar/error-wrapping
...
Remove repetitive error wrapping from internal/cache
2020-10-06 15:19:00 +02:00
1579d2a8ec
Remove some unused assignments
2020-10-06 14:55:13 +02:00
eba5dd831f
Fix typos reported by misspell
2020-10-06 14:55:13 +02:00
9ffb698c8d
Modernize context import in internal/fuse
...
Migrated using `go fix`.
2020-10-06 14:55:13 +02:00
efbb850d92
Remove a few redundant type specifiers
...
This is the result of running `gofmt -s -w **/*.go`
2020-10-06 14:55:13 +02:00
cfd57c480a
Merge pull request #2991 from restic/nonsense
...
cache/check: Clarify error message when given arguments
2020-10-06 00:40:12 +02:00
Michael Eischer
Alexander Neumann
rawtaz
greatroar
Hugo Gonzalez Labrador
Lorenz Bausch
Alexander Neumann
Alexander Weiss