Commit Graph

1427 Commits

Author SHA1 Message Date
Michael Eischer 306a29980a Print stacktrace in SIGINT handler if RESTIC_DEBUG_STACKTRACE_SIGINT set
The builtin mechanism to capture a stacktrace in Go is to send a SIGQUIT
to the running process. However, this mechanism is not avaiable on
Windows. Thus, tweak the SIGINT handler to dump a stacktrace if the
environment variable `RESTIC_DEBUG_STACKTRACE_SIGINT` is set.
2023-04-23 15:50:40 +02:00
Michael Eischer 22562d2132
Merge pull request #4300 from MichaelEischer/less-flaky-tests
Increase timeouts for lock refresh tests
2023-04-23 11:24:59 +02:00
Michael Eischer 51d823348d
Merge pull request #4286 from MichaelEischer/backend-cleanup-logging
Normalize backend logging and connection limiting
2023-04-23 11:24:26 +02:00
Michael Eischer 179e11c2ae Increase timeouts for lock refresh tests
When saving files to the local backend, in some cases the used fsync
calls are slow enough to cause the tests to time out. Thus, increase the
test timeouts as a stopgap measure until we can use the mem backend for
these tests.
2023-04-22 12:45:59 +02:00
Michael Eischer ebba233a3a backend/sema: rename constructor to NewBackend 2023-04-22 12:32:57 +02:00
Michael Eischer 1dd873b706
Merge pull request #4293 from MichaelEischer/fix-groups-by
Fix parent snapshot selection for relative paths
2023-04-21 22:47:03 +02:00
Michael Eischer 7a60d9e54f
Merge pull request #4288 from MichaelEischer/log-warnings-to-debug-log
Add warnings via Warnf to the debug log
2023-04-21 22:39:00 +02:00
Michael Eischer 3001dd8c2b Add test to verify that the backup parent is correctly selected 2023-04-21 22:35:02 +02:00
Michael Eischer 09cddb8927 rewrite: log snapshot saved before removal of the old snapshot
The snapshot was already saved before removing the old snapshot. Only
the log messages were printed in the wrong order.
2023-04-17 21:00:45 +02:00
Michael Eischer 913eab3361
Merge pull request #4234 from thndrbrrr/forget-opts-neg1-means-forever-issue-2565
restic forget --keep-* options will interpret -1 as "forever"
2023-04-14 23:18:47 +02:00
Michael Eischer a9c7c12276
Merge pull request #4265 from HeikoSchlittermann/chg/progress-to-stderr
generate: do not write progress to STDOUT
2023-04-14 23:18:23 +02:00
Michael Eischer 85eef232e6
Merge pull request #4232 from thndrbrrr/copy-chunker-params-verbose-msg
init: Add --copy-chunker-params verbose msg
2023-04-14 23:17:56 +02:00
Michael Eischer c934c99d41 gs: replace usage of context.Background() 2023-04-14 22:32:15 +02:00
Michael Eischer 8e1e3844aa backend: factor out connection limiting and parameter validation
The SemaphoreBackend now uniformly enforces the limit of concurrent
backend operations. In addition, it unifies the parameter validation.

The List() methods no longer uses a semaphore. Restic already never runs
multiple list operations in parallel.

By managing the semaphore in a wrapper backend, the sections that hold a
semaphore token grow slightly. However, the main bottleneck is IO, so
this shouldn't make much of a difference.

The key insight that enables the SemaphoreBackend is that all of the
complex semaphore handling in `openReader()` still happens within the
original call to `Load()`. Thus, getting and releasing the semaphore
tokens can be refactored to happen directly in `Load()`. This eliminates
the need for wrapping the reader in `openReader()` to release the token.
2023-04-14 22:32:15 +02:00
Michael Eischer 4703473ec5 backend: extract most debug logs into logger backend 2023-04-14 22:32:15 +02:00
Michael Eischer ba16904eed backup: Add test to verify parent snapshot selection for relative paths 2023-04-14 22:21:43 +02:00
Michael Eischer fab4a8a4d2 Properly initialize the --group-by option for backup tests 2023-04-14 21:53:01 +02:00
Torben Giesselmann 6d6c04abef forget: Simplify usage text 2023-04-14 10:05:23 -07:00
Heiko Schlittermann (HS12-RIPE) 087cf7e114
generate: write progress to STDOUT if this is a terminal
This allows usage as:

  eval "$(restic generated --bash-completion /dev/stdout)"
2023-04-12 22:39:59 +02:00
Michael Eischer e604939e72 Debug log status code if execution is interrupted
Currently, there is no clear indication in the debug log whether restic
exited or not.
2023-04-12 21:58:35 +02:00
Michael Eischer 37aca6bec0 Add warnings via Warnf to the debug log 2023-04-12 21:37:37 +02:00
Torben Giesselmann 71537da4b0 init: Add --copy-chunker-params verbose msg 2023-04-09 14:59:07 -07:00
Torben Giesselmann ce51d2f3c0 forget: Update usage text 2023-04-09 12:59:15 -07:00
Michael Eischer 1f43003cc1 Merge branch 'master' into feature-wait-for-unlock 2023-04-07 20:32:21 +02:00
Michael Eischer 8ce5f29758 lock: increase test timeout tolerances to avoid test failures 2023-04-07 20:07:05 +02:00
Michael Eischer cf1cc1fb72 lock: Print stacktrace if TestLockSuccessfulRefresh fails 2023-04-07 19:51:31 +02:00
Michael Eischer 64233ca0a7 lock: Improve debug logging in the test 2023-04-07 19:43:00 +02:00
jo ea59896bd6
Add a global option --retry-lock
Fixes restic#719

If the option is passed, restic will wait the specified duration of time
and retry locking the repo every 10 seconds (or more often if the total
timeout is relatively small).

- Play nice with json output
- Reduce wait time in lock tests
- Rework timeout last attempt
- Reduce test wait time to 0.1s
- Use exponential back off for the retry lock
- Don't pass gopts to lockRepo functions
- Use global variable for retry sleep setup
- Exit retry lock on cancel
- Better wording for flag help
- Reorder debug statement
- Refactor tests
- Lower max sleep time to 1m
- Test that we cancel/timeout in time
- Use non blocking sleep function
- Refactor into minDuration func

Co-authored-by: Julian Brost <julian@0x4a42.net>
2023-04-07 16:24:14 +02:00
Mark Herrmann f875a8843d restore: Add progress bar
Co-authored-by: Mark Herrmann <mark.herrmann@mailbox.org>
2023-04-07 12:08:23 +02:00
Torben Giesselmann 84ede6ad7a forget: Prevent neg. values in --keep-within* opts 2023-03-14 19:20:03 -07:00
Torben Giesselmann 667536cea4 forget: Allow neg. values in "--keep-within*" opts 2023-03-05 14:18:08 -08:00
Torben Giesselmann b77b0749fa forget: Treat -1 as forever for all "last n" opts 2023-03-04 19:01:37 -08:00
Torben Giesselmann 6aca7dac21 forget: Verify forget opts 2023-03-03 19:10:11 -08:00
Ben Wiederhake 9888443f5c check: Document behavior of --with-cache a bit better
Also see https://forum.restic.net/t/how-to-speed-up-tiny-incremental-checks/5905
2023-02-19 17:45:47 +01:00
greatroar a7786c67f1 cmd, restic: Refactor and fix snapshot filtering
This turns snapshotFilterOptions from cmd into a restic.SnapshotFilter
type and makes restic.FindFilteredSnapshot and FindFilteredSnapshots
methods on that type. This fixes #4211 by ensuring that hosts and paths
are named struct fields instead of unnamed function arguments in long
lists of such.

Timestamp limits are also included in the new type. To avoid too much
pointer handling, the convention is that time zero means no limit.
That's January 1st, year 1, 00:00 UTC, which is so unlikely a date that
we can sacrifice it for simpler code.
2023-02-19 15:04:25 +01:00
Michael Eischer 0ce182f044 document backup --group-by 2023-02-11 00:55:11 +01:00
Michael Eischer 2885db7902 backup: add group-by option 2023-02-10 23:18:14 +01:00
Michael Eischer acb40d2b94 Refactor group-by to parse options into a struct 2023-02-10 23:18:14 +01:00
Michael Eischer 2d700c3887 Make help for --verbose less confusing
The output is now
```
-v, --verbose                    be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)
```
instead of
```
-v, --verbose n                  be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)
```
2023-02-06 22:11:21 +01:00
greatroar d129baba7a repository: Reuse buffers in Repository.LoadUnpacked
This method had a buffer argument, but that was nil at all call sites.
That's removed, and instead LoadUnpacked now reuses whatever it
allocates inside its retry loop.
2023-01-30 22:01:01 +01:00
Alexander Neumann febb32b5b4 Set development version for 0.15.1 2023-01-30 20:43:52 +01:00
Alexander Neumann 7d4b7ad9cb Add version for 0.15.1 2023-01-30 20:43:47 +01:00
Michael Eischer 3b24e0ac55
Merge pull request #4154 from MichaelEischer/cleanup-prune-tests
deduplicate prune tests a bit
2023-01-26 22:39:15 +01:00
Alexander Neumann 65923e9c26
Merge pull request #4152 from MichaelEischer/invalid-locks
Improve handling of invalid locks
2023-01-25 08:19:53 +01:00
Michael Eischer 00216d54a1 deduplicate prune tests a bit 2023-01-14 19:05:45 +01:00
Michael Eischer 1f3f68b2c0 prune: Don't show "packs processed" for quiet runs 2023-01-14 18:48:32 +01:00
Michael Eischer 20ad14e362 lock: add help message how to recover from invalid locks 2023-01-14 18:04:22 +01:00
Michael Eischer c995b5be52 lock: cleanup error message
The error message is now `Fatal: unable to create lock in backend:
[...]` instead of `unable to create lock in backend: Fatal: [...]`.
2023-01-14 17:57:02 +01:00
Michael Eischer 4a7a6b06af ui/backup: Use progress.Updater for progress updates 2023-01-14 01:20:43 +01:00
Michael Eischer e499bbe3ae progress: extract progress updating into Updater struct
This allows reusing the code to create periodic progress updates.
2023-01-14 01:13:08 +01:00