Michael Eischer
660679c2f6
Merge pull request #4835 from MichaelEischer/fix-list-cancel
...
backend/retry: do not log final error if context was canceled
2024-06-01 19:17:30 +02:00
Michael Eischer
456ebfb0c7
Merge pull request #4840 from restic/dependabot/go_modules/google.golang.org/api-0.182.0
...
build(deps): bump google.golang.org/api from 0.181.0 to 0.182.0
2024-06-01 13:06:49 +00:00
Michael Eischer
830e87059a
Merge pull request #4842 from restic/dependabot/github_actions/docker/login-action-3.2.0
...
build(deps): bump docker/login-action from 3.1.0 to 3.2.0
2024-06-01 13:06:46 +00:00
Michael Eischer
a26beb9c1c
Merge pull request #4843 from restic/dependabot/github_actions/golangci/golangci-lint-action-6
...
build(deps): bump golangci/golangci-lint-action from 5 to 6
2024-06-01 13:06:43 +00:00
dependabot[bot]
46b0fac11e
build(deps): bump golangci/golangci-lint-action from 5 to 6
...
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action ) from 5 to 6.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases )
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v5...v6 )
---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-01 01:48:46 +00:00
dependabot[bot]
fced29b479
build(deps): bump docker/login-action from 3.1.0 to 3.2.0
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](e92390c5fb...0d4c9c5ea7
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-01 01:48:43 +00:00
dependabot[bot]
05d7885a87
build(deps): bump google.golang.org/api from 0.181.0 to 0.182.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.181.0 to 0.182.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.181.0...v0.182.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-06-01 01:02:01 +00:00
Michael Eischer
38654a3bd7
backend/retry: do not log final error if context was canceled
...
Calls to `List(ctx, ...)` are usually stopped by canceling the context
once no further entries are required by the caller. Thus, don't log the
final error if the used context was canceled.
2024-05-30 18:48:52 +02:00
Michael Eischer
2280fbfd2e
Merge pull request #4810 from konidev20/fix-gh-4768-add-custom-user-agent-for-http-client
...
Allow custom User-Agent to be specified for outgoing requests
2024-05-30 16:34:52 +00:00
Michael Eischer
7e0ee5974f
Merge pull request #4816 from MichaelEischer/skip-if-unchanged
...
backup: add support for `--skip-if-unchanged`
2024-05-30 15:39:08 +02:00
Srigovind Nayak
de7b418bbe
http: allow custom `User-Agent` for outgoing HTTP requests
2024-05-30 15:38:06 +02:00
Michael Eischer
c103c0830e
add changelog for skip-if-unchanged
2024-05-30 15:25:44 +02:00
Michael Eischer
513135b238
doc: slightly modernize backup examples
2024-05-30 15:25:44 +02:00
Michael Eischer
cdd210185d
Merge pull request #4823 from MichaelEischer/fix-check-output
...
Fix mangled check output
2024-05-30 15:17:27 +02:00
Michael Eischer
5649334099
backup: document --skip-if-unchanged
2024-05-30 15:14:32 +02:00
Michael Eischer
6869bdaaa8
backup: implement --skip-if-unchanged
2024-05-30 15:14:31 +02:00
Michael Eischer
13127f3f6b
check: remove output mangling workaround
2024-05-30 15:09:27 +02:00
Michael Eischer
939b537c80
check/migrate: convert to use termstatus to prevent mangled output
...
Errors reported by check would result in corrupted output.
2024-05-30 15:09:27 +02:00
Michael Eischer
7b4f81d964
Merge pull request #4829 from greatroar/termstatus
...
termstatus: Do formatting in Message, not Terminal
2024-05-30 11:57:44 +00:00
Michael Eischer
a5c7a83470
Merge pull request #4819 from restic/update-dependencies
...
Update dependencies
2024-05-30 13:55:46 +02:00
Michael Eischer
52b3fee42b
Merge pull request #4818 from MichaelEischer/move-code
...
Move cache / index / hashing / pack packages
2024-05-30 13:54:31 +02:00
Michael Eischer
e95cfff923
Merge pull request #4826 from MichaelEischer/fix-docs-warninigs
...
doc: fix warnings
2024-05-30 13:53:49 +02:00
Michael Eischer
fe8fbcc5ed
Merge pull request #4822 from MichaelEischer/consistent-backup-source-name
...
backup: refer to backed up data as backup source not target
2024-05-30 13:53:25 +02:00
Michael Eischer
3d4a620089
Merge pull request #4824 from MichaelEischer/fix-cache-race
...
bloblru: Fix flaky test due to race condition
2024-05-30 13:52:34 +02:00
greatroar
46f04bc5f0
termstatus: Do formatting in Message, not Terminal
2024-05-27 19:05:39 +02:00
Michael Eischer
a651af70d6
doc: fix warings
2024-05-26 14:32:09 +02:00
Michael Eischer
1c6067d93d
bloblru: variable name cleanup
2024-05-26 12:42:46 +02:00
Michael Eischer
21ce03cff2
bloblru: move defer outside critical section
2024-05-26 12:38:20 +02:00
Michael Eischer
0ad0b7ca7c
bloblru: fix race condition that can compute value multiple times
2024-05-26 12:37:24 +02:00
Michael Eischer
619ce8bfa2
Merge pull request #4820 from MichaelEischer/forget-extend-descritpion
...
forget: explain oldest snapshot special case in command help
2024-05-26 10:04:43 +00:00
Michael Eischer
107f18849c
doc: update help output in manual_rest.rst
2024-05-25 23:16:22 +02:00
Michael Eischer
18990dbed8
backup: refer to backed up data as backup source not target
...
A backup reads from the source and writes it to the backup target
(repository). The name "target" is kept within the source code as it's
used there excessively.
2024-05-25 23:13:37 +02:00
Michael Eischer
3828313974
Merge pull request #4821 from MichaelEischer/master
...
update `repair packs` changelog
2024-05-25 17:02:21 +02:00
Michael Eischer
04c181dbd0
update-repair-packs-changelog
2024-05-25 16:42:55 +02:00
Michael Eischer
21528c9f02
Merge pull request #4805 from plant99/mkdirall-cachedir
...
prepareCheckCache function: Add MkdirAll() to restic-cache directory if it doesn't exist
2024-05-25 14:30:21 +00:00
Shivashis Padhi
0271bb97f8
check: enhance check command to create cache directory if it doesn't exist
2024-05-25 19:15:42 +05:30
Michael Eischer
2c7ebf4c7f
forget: explain oldest snapshot special case in command help
2024-05-25 15:10:21 +02:00
Michael Eischer
7ea508e7b8
Merge pull request #4815 from greatroar/termstatus
...
Termstatus refactoring
2024-05-25 12:08:53 +00:00
Michael Eischer
496e57f956
hashing: move to repository package
2024-05-25 13:13:03 +02:00
Michael Eischer
5e0ea8fcfa
pack: move to repository package
2024-05-25 13:13:03 +02:00
Michael Eischer
50ec408302
index: move to repository package
2024-05-25 13:13:03 +02:00
Michael Eischer
58bbb27b4c
downgrade minio to keep go 1.19 support
2024-05-25 12:52:08 +02:00
Michael Eischer
c5260373ca
update backoff/xxhash libraries
2024-05-25 12:37:14 +02:00
Michael Eischer
b2f7a878cd
update backend libraries
...
minio-go is kept at 7.0.67 as later versions require Go 1.21.
2024-05-25 12:36:16 +02:00
Michael Eischer
f20bf5eeba
update golang.org/x packages
2024-05-25 12:35:28 +02:00
Michael Eischer
8e5d7d719c
cache: move to backend package
2024-05-24 23:04:06 +02:00
Michael Eischer
80132e71d8
Merge pull request #4808 from MichaelEischer/insecure-no-password
...
Implement `--insecure-no-password` option.
2024-05-24 22:58:25 +02:00
Michael Eischer
130506250f
document insecure-no-password
2024-05-24 22:38:20 +02:00
Michael Eischer
1d2277b4c3
Add --insecure-no-password option
...
This also includes two derived options `--from-insecure-no-password`
used for commands that require specifying a source repository. And
`--new-insecure-no-password` for the `key add` and `key passwd`
commands.
Specifying `--insecure-no-password` disabled the password prompt and
immediately uses an empty password. Passing a password via CLI option or
environment variable at the same time is an error.
2024-05-24 22:38:20 +02:00
Michael Eischer
d4b0d21199
key add/passwd: deduplicate options setup and remove globals
...
The current pattern of using a global options variable is problematic.
2024-05-24 22:38:20 +02:00
Michael Eischer
55cb8d174a
Merge pull request #4354 from MichaelEischer/associated-data
...
Reduce prune memory usage
2024-05-24 22:26:22 +02:00
Michael Eischer
436afbff23
add changelog for memory efficient prune
2024-05-24 22:18:14 +02:00
Michael Eischer
3c7b7efdc9
repository: remove prune plan parts once they are no longer necessary
2024-05-24 22:18:14 +02:00
Michael Eischer
462b82a060
index: reduce size of compressed indexes
...
use the same index size for compressed and uncompressed indexes.
Otherwise, decoding the index of a compressed repository requires
significantly more memory.
2024-05-24 22:18:14 +02:00
Michael Eischer
77873f5a9d
repository: let prune control data structure of usedBlobs set
2024-05-24 22:18:14 +02:00
Michael Eischer
2033c02b09
index: replace CountedBlobSet with AssociatedSet
2024-05-24 22:18:14 +02:00
Michael Eischer
93098e9265
prune: hide implementation details of counted blob set
2024-05-24 21:42:56 +02:00
Michael Eischer
ff4775a15f
Merge pull request #4812 from MichaelEischer/streaming-index-rewrite
...
Resumable prune & memory-efficient index rewrite
2024-05-24 21:41:30 +02:00
Michael Eischer
860b595a8b
backend: increase watchdog test timeout for deflaking
2024-05-24 21:33:17 +02:00
Michael Eischer
f680a2331d
add changelog for streaming index rewrite
2024-05-24 21:33:17 +02:00
Michael Eischer
027cc64737
repository: fix prune heuristic to allow resuming interrupted runs
...
Pack files created by interrupted prune runs, appear to consist only of
duplicate blobs on the next run. This caused the previous heuristic to
ignore those pack files. Now, a duplicate blob in a specific pack file
is also selected if that pack file only contains duplicate blobs. This
allows prune to select the already rewritten pack files.
2024-05-24 21:33:17 +02:00
Michael Eischer
e52033a8bd
index: slightly reduce Rewrite concurrency
...
The index operations are likely CPU-bounded. Thus, reduce the
concurrency accordingly.
2024-05-24 21:33:17 +02:00
Michael Eischer
57d69aa640
index: cleanup SaveIndex method
2024-05-24 21:33:17 +02:00
Michael Eischer
2ca1c37721
index: additional tests for new index save methods
2024-05-24 21:33:17 +02:00
Michael Eischer
5f7b48e65f
index: replace Save() method with Rewrite and SaveFallback
...
Rewrite implements a streaming rewrite of the index that excludes the
given packs. For this it loads all index files from the repository and
only modifies those that require changes. This will reduce the index
churn when running prune. Rewrite does not require the in-memory index
and thus can drop it to significantly reduce the memory usage.
However, `prune --unsafe-recovery` cannot use this strategy and requires
a separate method to save the whole in-memory index. This is now handled
using SaveFallback.
2024-05-24 21:33:17 +02:00
Michael Eischer
ad98fbf7dd
restic: add IDSet.Clone() method
2024-05-24 21:33:17 +02:00
Michael Eischer
72482ce5bd
index: misc cleanups
2024-05-24 21:33:17 +02:00
Michael Eischer
68fa0e0305
prune: no longer disable automatic index updates
...
this allows prune to resume an interrupted prune run.
2024-05-24 21:33:17 +02:00
Michael Eischer
9aa0c90fb2
index: remove supersedes field
...
Using the field with its current semantics is nearly impossible to get
right. Remove it as it will be replaced anyways in repository format 3.
2024-05-24 21:33:17 +02:00
Michael Eischer
76e6719f2e
repository: make CreateIndexFromPacks method private
2024-05-24 21:33:17 +02:00
Michael Eischer
04ad9f0c0c
repository: remove Packer and SavePacker from public interface
2024-05-24 21:33:17 +02:00
Michael Eischer
550d1eeac3
repository: remove SaveIndex from interface
...
The method is now only indirectly accessible via Prune or RepairIndex.
2024-05-24 21:33:17 +02:00
Michael Eischer
fb59e00614
index: rewrite MasterIndex load/save test to be independent of repository
2024-05-24 21:33:17 +02:00
Michael Eischer
447b486c20
index: deduplicate index loading of check and repository
2024-05-24 21:33:17 +02:00
Michael Eischer
6ca12c1b4a
archiver: replace most uses of restic.Repository
2024-05-24 21:33:17 +02:00
Michael Eischer
864995271e
repository: unwrap BlobHandle parameters of LookupBlob
...
The method now uses the same parameters as LookupBlobSize.
2024-05-24 21:33:17 +02:00
Michael Eischer
1266a4932f
repository: fix parameter order of LookupBlobSize
...
All methods should use blobType followed by ID.
2024-05-24 21:33:17 +02:00
Michael Eischer
e848ad651a
restic: name parameters in restic interface
2024-05-24 21:33:17 +02:00
Michael Eischer
0bb0720348
test cleanups
2024-05-24 21:33:17 +02:00
Michael Eischer
c01bcb1001
archiver: remove unused masterIndex from test
2024-05-24 21:33:17 +02:00
Michael Eischer
0aa5c53842
repository: replace HasBlob with LookupBlobSize
2024-05-24 21:33:17 +02:00
Michael Eischer
8f1e70cd9b
repository: remove clearIndex and packSize from public interface
2024-05-24 21:33:17 +02:00
Michael Eischer
4df887406f
repository: inline MasterIndex interface into Repository interface
2024-05-24 21:33:17 +02:00
Michael Eischer
3eeb6723cd
Merge pull request #4764 from MichaelEischer/safe-keep-tag
...
Prevent unsafe uses of `forget --keep-tag`
2024-05-24 20:51:20 +02:00
Michael Eischer
3cc6827f09
forget: add feature flag for safe --keep-tags behavior
2024-05-24 20:45:33 +02:00
Michael Eischer
7948912b31
document --unsafe-allow-remove-all option
2024-05-24 20:45:33 +02:00
Michael Eischer
3f46808898
add forget safety net changelogs
2024-05-24 20:45:33 +02:00
Michael Eischer
82ca0030b7
forget: test --unsafe-allow-remove-all and --keep-tags safety check
2024-05-24 20:45:33 +02:00
Michael Eischer
57f9739573
forget: Add --unsafe-allow-remove-all option
...
To prevent accidentally wiping all snapshots from a repository, that
option can only be used if either a snapshot filter or a keep policy is
specified.
Essentially, the option allows `forget --tag something
--unsafe-allow-remove-all` calls to remove all snapshots with a specific
tag.
2024-05-24 20:45:33 +02:00
Michael Eischer
5b7952e426
forget: return error if no policy was specified
2024-05-24 20:45:33 +02:00
Michael Eischer
c0e1f36830
forget: refuse deleting the last snapshot in a snapshot group
...
`--keep-tag invalid-tag` was previously able to wipe all snapshots in a
repository. As a user specified a `--keep-*` option this is likely
unintentional. This forbid deleting all snapshot if a `--keep-*` option
was specified to prevent data loss. (Not specifying such an option
currently also causes the command to abort)
2024-05-24 20:45:33 +02:00
Michael Eischer
d106ad6921
restic: regenerate snapshot keep policy golden test files
2024-05-24 20:45:33 +02:00
Michael Eischer
16ef4d515b
Merge pull request #4784 from MichaelEischer/rework-backend-retries
...
Rework backend retries
2024-05-24 20:29:54 +02:00
Michael Eischer
e4a48085ae
backend/retry: feature flag new retry behavior
2024-05-24 20:24:02 +02:00
Michael Eischer
723247c8e5
add changelog for longer retries
2024-05-24 20:24:02 +02:00
Michael Eischer
b1266867d2
repository: wait max 1 minutes for lock removal if context is canceled
...
The toplevel context in restic only canceled if the user interrupts a
restic operation. If the network connection has failed this can require
waiting the full retry duration of 15 minutes which is a bad user
experience for interactive usage. Thus limit the delay to one minute in
this case.
2024-05-24 20:24:02 +02:00
Michael Eischer
98709a4372
retry: reduce total number of retries
...
Retries in restic try to solve two main problems:
- retry a temporarily failed operation
- tolerate temporary network interruptions
The first problem only requires a few retries, whereas the last one benefits
primarily from spreading the requests over a longer duration.
Increasing the default multiplier and the initial interval works for
both cases. The first few retries only take a few seconds, while later
retries quickly reach the maximum interval of one minute. This ensures
that the total number of retries issued by restic will remain at around
21 retries for a 15 minute period. As the concurrency in restic is
bounded, retries drastically reduce the number of requests sent to a
backend. This helps to prevent overloading the backend.
2024-05-24 20:24:02 +02:00
Michael Eischer
512cd6ef07
retry: ensure that there's always at least one retry
...
Previously, if an operation failed after 15 minutes, then it would never
be retried. This means that large backend requests are more unreliable
than smaller ones.
2024-05-24 20:24:02 +02:00
Michael Eischer
a60ee9b764
retry: limit retries based on elapsed time not count
...
Depending on how long an operation takes to fail, the total retry
duration can currently vary between 1.5 and 15 minutes. In particular
for temporarily interrupted network connections, the former timeout is
too short. Thus always use a limit of 15 minutes.
2024-05-24 20:24:02 +02:00
Michael Eischer
a3633cad9e
retry: explicitly log failed requests
...
This simplifies finding the request in the log output that cause an
operation to fail.
2024-05-24 20:24:02 +02:00
Michael Eischer
b9cbf623fa
Merge pull request #4814 from MichaelEischer/fix-tmp-docs
...
doc: fix tmpdir documentation for windows
2024-05-24 18:17:59 +02:00
greatroar
66d03c797e
ui, termstatus: Move WrapStdio
...
Saves some imports. ui still needs to import ui/termstatus from
message.go.
2024-05-21 11:25:16 +02:00
greatroar
0b56214473
ui: Simplify stdio wrapper
...
The StdioWrapper type is really just a pair of io.WriteClosers, so
remove it in favor of a function that returns two of those. Test
coverage increases because the removed code was not tested.
2024-05-21 11:23:32 +02:00
greatroar
7f439a9c34
ui: Inline lineWriter into StdioWrapper
2024-05-21 11:00:09 +02:00
Michael Eischer
4021e67d97
doc: fix tmpdir documentation for windows
2024-05-20 20:48:29 +02:00
Michael Eischer
8898f61717
Merge pull request #4809 from MichaelEischer/update-changelog
...
add retries for corrupted blobs to changelog
2024-05-18 23:04:13 +02:00
Michael Eischer
5f23baabcc
add retries for corrupted blobs to changelog
2024-05-18 23:03:24 +02:00
Michael Eischer
9c5bac6f25
Merge pull request #4799 from letmaik/letmaik/azure-force-cli-credential
...
Azure: add option to force use of CLI credential
2024-05-18 20:22:15 +00:00
Michael Eischer
c56ecec9aa
azure: deduplicate cli and default credentials case
2024-05-18 22:15:54 +02:00
Maik Riechert
355f520936
Azure: add option to force use of CLI credential
2024-05-18 22:15:54 +02:00
Michael Eischer
1dfe1b8732
Merge pull request #4802 from MichaelEischer/backend-cleanups
...
Repository: Remove Backend() method
2024-05-18 22:02:45 +02:00
Michael Eischer
223aa22cb0
replace some uses of restic.Repository with finegrained interfaces
2024-05-18 21:42:51 +02:00
Michael Eischer
291c9677de
restic/repository: remove Backend() method
2024-05-18 21:42:51 +02:00
Michael Eischer
673496b091
repository: clean cache between CheckPack retries
...
The cache cleanup pattern is also used in ListPack etc.
2024-05-18 21:42:51 +02:00
Michael Eischer
3d2410ed50
Replace some repo.RemoveUnpacked usages
...
These will eventually be blocked as they do not delete Snapshots.
2024-05-18 21:42:51 +02:00
Michael Eischer
d2c26e33f3
repository: remove further usages of repo.Backend()
2024-05-18 21:42:51 +02:00
Michael Eischer
8a425c2f0a
remove usages of repo.Backend() from tests
2024-05-18 21:42:51 +02:00
Michael Eischer
aa4647f773
repository: unexport PackBlobIterator
2024-05-18 21:42:51 +02:00
Michael Eischer
94e863885c
check: move verification of individual pack file to repository
2024-05-18 21:42:50 +02:00
Michael Eischer
e40943a75d
restic: remove backend usage from lock test
2024-05-18 21:38:31 +02:00
Michael Eischer
67e2ba0d40
repository: Lock requires *repository.Repository
...
This allows the Lock function to access the backend, even once the
Backend method is removed from the interface.
2024-05-18 21:38:31 +02:00
Michael Eischer
d8b184b3d3
repository: convert test helper to return *repository.Repository
2024-05-18 21:38:31 +02:00
Michael Eischer
a1ca5e15c4
migrations: add temporary hack for s3_layout
...
The migration will be removed after the next restic release anyways.
Thus, there's no need for a clean implementation.
2024-05-18 21:38:31 +02:00
Michael Eischer
34d90aecf9
migrations: move logic of upgrade_repo_v2 to repository package
...
The migration modifies repository internals and thus should live within
the repository package.
2024-05-18 21:38:31 +02:00
Michael Eischer
ab9077bc13
replace usages of backend.Remove() with repository.RemoveUnpacked()
...
RemoveUnpacked will eventually block removal of all filetypes other than
snapshots. However, getting there requires a major refactor to provide
some components with privileged access.
2024-05-18 21:38:31 +02:00
Michael Eischer
8274f5b101
prune: remove Backend.IsNotExist()
...
Only handling one specific error is not particularly useful.
2024-05-18 21:38:31 +02:00
Michael Eischer
9795198189
debug: remove Backend.Stat() usage
2024-05-18 21:38:31 +02:00
Michael Eischer
0c1ba6d95d
backend: remove unused Location method
2024-05-18 21:38:31 +02:00
Michael Eischer
eb6c653f89
Merge pull request #4800 from MichaelEischer/cleanup-load
...
Retry loading of corrupted data from backend / cache
2024-05-18 21:34:54 +02:00
Michael Eischer
74d90653e0
check: use ReadFull to load pack header in checkPack
...
This ensures that the pack header is actually read completely.
Previously, for a truncated file it was possible to only read a part of
the header, as backend.Load(...) is not guaranteed to return as many
bytes as requested by the length parameter.
2024-05-18 21:28:54 +02:00
Michael Eischer
8f8d872a68
fix compatibility with go 1.19
2024-05-18 21:28:54 +02:00
Michael Eischer
ff0744b3af
check: test checkPack retries
2024-05-18 21:28:54 +02:00
Michael Eischer
987c3b250c
repository: test retries of ListPack
2024-05-18 21:28:54 +02:00
Michael Eischer
bf16096771
repository: test LoadBlob retries
2024-05-18 21:28:54 +02:00
Michael Eischer
4f45668b7c
repository: rework and extend LoadRaw tests
2024-05-18 21:28:54 +02:00
Michael Eischer
ac805d6838
cache: cleanup debug logs
2024-05-18 21:28:54 +02:00
Michael Eischer
5214af88e2
cache: test forget behavior
2024-05-18 21:28:54 +02:00
Michael Eischer
3ff063e913
check: verify pack a second time if broken
2024-05-18 21:28:54 +02:00
Michael Eischer
385cee09dc
repository: fix caching of tree packs in LoadBlobsFromPack
2024-05-18 21:28:54 +02:00
Michael Eischer
e734746f75
cache: forget cached file at most once
...
This is inspired by the circuit breaker pattern used for distributed
systems. If too many requests fails, then it is better to immediately
fail new requests for a limited time to give the backend time to
recover.
By only forgetting a file in the cache at most once, we can ensure that
a broken file is only retrieved once again from the backend. If the file
stored there is broken, previously it would be cached and deleted
continuously. Now, it is retrieved only once again, all later requests
just use the cached copy and either succeed or fail immediately.
2024-05-18 21:28:54 +02:00
Michael Eischer
97a307df1a
cache: Always use cached file if it exists
...
A file is always cached whole. Thus, any out of bounds access will also
fail when directed at the backend. To handle case in which the cached
file is broken, then caller must call Cache.Forget(h) for the file in
question.
2024-05-18 21:28:54 +02:00
Michael Eischer
8cce06d915
repair packs: drop experimental warning
...
This warning should already have been removed once the feature flag was
dropped.
2024-05-18 21:28:54 +02:00
Michael Eischer
433a6aad29
repository: remove redundant blob loading fallback from RepairPacks
...
LoadBlobsFromPack already implements the same fallback behavior.
2024-05-18 21:28:54 +02:00
Michael Eischer
e401af07b2
check: fix error message formatting
2024-05-18 21:28:54 +02:00
Michael Eischer
7017adb7e9
repository: retry failed ListPack once
2024-05-18 21:28:54 +02:00
Michael Eischer
e33ce7f408
repository: retry failed LoadBlob once
2024-05-18 21:28:54 +02:00
Michael Eischer
2ace242f36
repository: make reloading broken files explicit
2024-05-18 21:28:54 +02:00
Michael Eischer
e9390352a7
cache: code cleanups
2024-05-18 21:26:00 +02:00
Michael Eischer
503c8140b1
repository: unify blob decoding code
2024-05-18 21:26:00 +02:00
Michael Eischer
6563f1d2ca
repository: remove redundant debug log
2024-05-18 21:26:00 +02:00
Michael Eischer
021fb49559
repository: Implement repository.LoadUnpacked using LoadRaw
...
Both functions were using a similar implementation.
2024-05-18 21:26:00 +02:00
Michael Eischer
779c8d3527
debug/repair packs/upgrade repo v2: use repository.LoadRaw
...
This replaces calling the low-level backend.Load() method.
2024-05-18 21:26:00 +02:00
Michael Eischer
1d6d3656b0
repository: move backend.LoadAll to repository.LoadRaw
...
LoadRaw also includes improved context cancellation handling similar to the
implementation in repository.LoadUnpacked.
The removed cache backend test will be added again later on.
2024-05-18 21:26:00 +02:00
Michael Eischer
47232bf8b0
backend: move LimitReadCloser to util package
...
The helper is only intended for usage by backend implementations.
2024-05-18 21:26:00 +02:00
Michael Eischer
dcd151147c
Merge pull request #4803 from restic/permanent-retry-failure
...
Do not retry permanent backend failures
2024-05-18 20:07:06 +02:00
Michael Eischer
53d15bcd1b
retry: add circuit breaker to load method
...
If a file exhausts its retry attempts, then it is likely not accessible
the next time. Thus, immediately fail all load calls for that file to
avoid useless retries.
2024-05-18 19:59:26 +02:00
Michael Eischer
394c8ca3ed
rest/rclone/s3/sftp/swift: move short file detection behind feature gate
...
These backends tend to use a large variety of server implementations.
Some of those implementations might prove problematic with the new
checks.
2024-05-18 19:59:26 +02:00
Michael Eischer
6328b7e1f5
replace "too small" with "too short" in error messages
2024-05-18 19:59:26 +02:00
Michael Eischer
53561474d9
update changelog with persistent backend error handling
2024-05-18 19:59:26 +02:00
Michael Eischer
aeb7eb245c
retry: do not retry permanent errors
...
This is currently gated behind a feature flag as some unexpected
interactions might show up in the wild.
2024-05-18 19:59:26 +02:00
Michael Eischer
bf8cc59889
Use generic backend-error-redesign feature flag instead of http-timeouts
...
An individual flag for each change of the backend error handling would
be too finegrained. Thus, add a generic flag.
2024-05-18 19:54:52 +02:00
Michael Eischer
4740528a0b
backend: add tests for IsPermanentError
2024-05-18 19:54:52 +02:00
Michael Eischer
6a85df7297
backend: add IsPermanentError() method to interface
2024-05-18 19:54:52 +02:00
Michael Eischer
cfc420664a
mem: stricter handling of out of bounds requests
2024-05-18 19:54:52 +02:00
Michael Eischer
d40f23e716
azure/b2/gs/s3/swift: adapt cloud backend
2024-05-18 19:54:51 +02:00
Michael Eischer
e793c002ec
local: stricter handling of short files
2024-05-18 19:54:21 +02:00
Michael Eischer
b4895ebd76
rest: rework error reporting and report too short files
2024-05-18 19:54:21 +02:00
Michael Eischer
eaa3f81d6b
sftp: check for truncated files without an extra backend request
2024-05-18 19:54:21 +02:00
Michael Eischer
c6d74458ee
sftp: improve handling of too short files
2024-05-18 19:54:21 +02:00
Michael Eischer
7ed560a201
Merge pull request #4796 from MichaelEischer/parallel-dump-load
...
dump: Parallelize loading large files
2024-05-14 22:35:44 +02:00
Michael Eischer
92221c2a6d
Merge pull request #4708 from zmanda/windows-securitydesc
...
Back up and restore SecurityDescriptors on Windows
2024-05-12 14:14:39 +00:00
Michael Eischer
b5fdb1d637
Merge pull request #4782 from MichaelEischer/fix-sftp-performance
...
Fix sftp upload performance
2024-05-12 15:28:33 +02:00
Michael Eischer
e4f9bce384
Merge pull request #4792 from restic/request-watchdog
...
backend: enforce that backend HTTP requests make progress
2024-05-09 23:55:30 +02:00
Michael Eischer
3740700ddc
add http timeouts to changelog
2024-05-09 23:46:17 +02:00
Michael Eischer
ebd01a4675
backend: add tests for watchdogRoundTripper
2024-05-09 23:46:17 +02:00
Michael Eischer
8778670232
backend: cancel stuck http requests
...
requests that make no upload or download progress within a timeout are
canceled.
2024-05-09 23:46:17 +02:00
Michael Eischer
0987c731ec
backend: configure protocol-level connection health checks
...
This should detect a connection that is stuck for more than 2 minutes.
2024-05-09 23:46:17 +02:00
aneesh-n
a4fd1b91e5
Fix review comments
...
Change lowerPrivileges from bool to atomic.Bool.
Add missing cleanup from upstream go-winio.
Add handling for ERROR_NOT_ALL_ASSIGNED warning.
2024-05-06 16:54:08 -06:00
Michael Eischer
e184538ddf
dump: add changelog
2024-05-05 12:12:21 +02:00
Michael Eischer
4d55a62ada
bloblru: add test for GetOrCompute
2024-05-05 12:00:25 +02:00
Michael Eischer
7cce667f92
fuse: switch to use bloblru.GetOrCompute
2024-05-05 11:38:42 +02:00
Michael Eischer
bd03af2feb
dump: add GetOrCompute to bloblru cache
2024-05-05 11:38:42 +02:00
Michael Eischer
45509eafc8
dump: load blobs of a file from repository in parallel
2024-05-05 11:38:42 +02:00
Michael Eischer
24c1822220
Merge pull request #4794 from flow-c/master
...
Update 060_forget.rst
2024-05-04 08:25:06 +00:00
flow-c
d4477a5a99
Update 060_forget.rst
...
Replace deprecated `-1` with `unlimited` in calendar-related `--keep-*` options
2024-05-04 09:32:25 +02:00
Michael Eischer
ffe5439149
Merge pull request #4605 from MichaelEischer/better-restorer-error-handling
...
Rework repository.StreamPacks & better restorer error handling
2024-05-01 16:37:41 +02:00
Michael Eischer
676f0dc60d
add changelog
2024-05-01 16:28:57 +02:00
Michael Eischer
1e57057953
Merge pull request #4789 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob-1.3.2
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 1.3.1 to 1.3.2
2024-05-01 10:45:47 +00:00
Michael Eischer
1ba0af6993
Merge pull request #4787 from restic/dependabot/go_modules/github.com/klauspost/compress-1.17.8
...
build(deps): bump github.com/klauspost/compress from 1.17.7 to 1.17.8
2024-05-01 10:44:33 +00:00
Michael Eischer
ffc41ae62a
Merge pull request #4786 from restic/dependabot/go_modules/golang.org/x/net-0.24.0
...
build(deps): bump golang.org/x/net from 0.23.0 to 0.24.0
2024-05-01 10:41:26 +00:00
Michael Eischer
4832c2fbfa
Merge pull request #4790 from restic/dependabot/github_actions/golangci/golangci-lint-action-5
...
build(deps): bump golangci/golangci-lint-action from 4 to 5
2024-05-01 10:37:37 +00:00
dependabot[bot]
30609ae6b2
build(deps): bump golangci/golangci-lint-action from 4 to 5
...
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action ) from 4 to 5.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases )
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v4...v5 )
---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-05-01 01:45:43 +00:00
dependabot[bot]
502e5867a5
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go ) from 1.3.1 to 1.3.2.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.3.1...sdk/storage/azblob/v1.3.2 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-05-01 01:02:39 +00:00
dependabot[bot]
18a6d6b408
build(deps): bump github.com/klauspost/compress from 1.17.7 to 1.17.8
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.17.7 to 1.17.8.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.17.7...v1.17.8 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-05-01 01:02:22 +00:00
dependabot[bot]
3bb88e8307
build(deps): bump golang.org/x/net from 0.23.0 to 0.24.0
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.23.0 to 0.24.0.
- [Commits](https://github.com/golang/net/compare/v0.23.0...v0.24.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-05-01 01:02:11 +00:00
aneesh-n
672f6cd776
Fix review comments for privileges and security flags
2024-04-29 17:29:51 -06:00
aneesh-n
08c6945d61
Fix review comments
2024-04-29 16:21:38 -06:00
Aneesh N
3f76b902e5
Merge branch 'master' into windows-securitydesc
2024-04-29 14:40:34 -06:00
Michael Eischer
ccac7c7fb3
Merge pull request #3067 from DRON-666/vss-options
...
Add options to fine tune VSS snapshots
2024-04-29 18:09:47 +00:00
DRON-666
ccd35565ee
s/sec./seconds
2024-04-29 01:48:22 +03:00
DRON-666
125dba23c5
Rearange code
2024-04-29 01:27:34 +03:00
DRON-666
7ee889bb0d
Use S_FALSE and MaxInt
2024-04-29 01:25:25 +03:00
DRON-666
90b168eb6c
isMountPointExcluded to isMountPointIncluded
2024-04-29 01:23:50 +03:00
DRON-666
24330c19a8
Use kebab case in option names
2024-04-29 01:21:33 +03:00
DRON-666
5703e5a652
Fix texts and comments
2024-04-29 01:18:46 +03:00
DRON-666
0a8f9c5d9c
vss: Add tests for "provider" option
2024-04-28 22:45:21 +03:00
DRON-666
739d3243d9
vss: Update docs and changelog
2024-04-28 22:45:21 +03:00
DRON-666
bb0f93ef3d
vss: Add "provider" option
2024-04-28 22:45:21 +03:00
DRON-666
3bac1f0135
vss: Fix issues reported by linters
2024-04-28 22:45:21 +03:00
DRON-666
88c509e3e9
vss: Change `ErrorHandler` signature
...
We don't need `error` here: the only existing implementation
of `ErrorHandler` always call `Backup.Error` and all
implementations of `Backup.Error` always return nil.
2024-04-28 22:44:16 +03:00
DRON-666
9d3d915e2c
vss: Add some tests
2024-04-28 22:44:16 +03:00
DRON-666
9182e6bab5
vss: Update docs and changelog
2024-04-28 22:44:16 +03:00
DRON-666
c4f67c0064
vss: Add volume filtering
...
Add options to exclude all mountpoints and arbitrary volumes from snapshotting.
2024-04-28 22:44:15 +03:00
DRON-666
7470e5356e
vss: Add "timeout" option
...
Changing multiple "callAsyncFunctionAndWait" with fixed timeout
to calculated timeout based on deadline.
2024-04-28 22:44:15 +03:00
DRON-666
78dbc5ec58
vss: Add initial support for extended options
2024-04-28 22:44:15 +03:00
Michael Eischer
a1d682ce0e
add changelog for sftp performance fix
2024-04-28 11:58:08 +02:00
Michael Eischer
935327d480
sftp: slightly increase write concurrency
...
This should increase upload throughput for high latency links a bit.
2024-04-28 11:50:09 +02:00
Michael Eischer
669a669603
sftp: Fix upload performance issue
...
Since pkg/sftp 1.13.0 files were uploaded sequentially using 32kb chunks
instead of sending 64 chunks in parallel.
2024-04-28 11:48:26 +02:00
Michael Eischer
faffd15d13
Merge pull request #4734 from maouw/enhancement/envvar-for-host
...
Add support for specifying --host via environment variable
2024-04-24 20:00:15 +00:00
Michael Eischer
347e9d0765
complete RESITC_HOST environment handling & test
2024-04-24 21:52:39 +02:00
Altan Orhon
871ea1eaf3
Add support for specifying --host via environment variable
...
This commit adds support for specifying the `--host` option via the `RESTIC_HOST` environment variable. This is done by extending option processing in `cmd_backup.go` and for `restic.SnapshotFilter` in `find.go`.
2024-04-24 21:49:42 +02:00
Michael Eischer
a7b5e09902
Merge pull request #4753 from MichaelEischer/remove-cleanup-handlers
...
Replace cleanup handlers with context based command cancelation
2024-04-24 21:34:19 +02:00
Michael Eischer
3f9d50865d
Merge pull request #4776 from MichaelEischer/cleanup-backend-open
...
unify backend open and create
2024-04-24 21:24:27 +02:00
Michael Eischer
5f263752d7
init: also apply limiter for non-HTTP backend
2024-04-24 20:42:30 +02:00
Michael Eischer
484dbb1cf4
get rid of a few global variables
2024-04-22 22:39:33 +02:00
Michael Eischer
940a3159b5
let index.Each() and pack.Size() return error on canceled context
...
This forces a caller to actually check that the function did complete.
2024-04-22 22:39:32 +02:00
Michael Eischer
31624aeffd
Improve command shutdown on context cancellation
2024-04-22 22:31:38 +02:00
Michael Eischer
910927670f
mount: fix exit code on cancellation
2024-04-22 22:27:19 +02:00
Michael Eischer
6f2a4dea21
remove global shutdown hook
2024-04-22 22:27:19 +02:00
Michael Eischer
699ef5e9de
debug: replace cleanup handler usage in profiling setup
2024-04-22 22:27:19 +02:00
Michael Eischer
eb710a28e8
use standalone shutdown hook for readPasswordTerminal
...
move terminal restoration into readPasswordTerminal
2024-04-22 22:27:19 +02:00
Michael Eischer
86c7909f41
mount: use standalone shutdown hook via goroutine
2024-04-22 22:27:19 +02:00
Michael Eischer
93135dc705
lock: drop cleanup handler
2024-04-22 22:27:19 +02:00
Michael Eischer
21a7cb405c
check: replace cleanup handler
2024-04-22 22:27:19 +02:00
Michael Eischer
b15d867414
Merge pull request #4763 from MichaelEischer/refactor-prune
...
Refactor repair index / prune into the repository package
2024-04-22 22:24:53 +02:00
Michael Eischer
2e6c43c695
Merge pull request #4761 from MichaelEischer/fix-cache-race
...
cache: ignore ErrNotExist during cleanup of old files
2024-04-22 21:46:06 +02:00
Michael Eischer
f7632de3d6
Merge pull request #4772 from MichaelEischer/better-error-on-too-large-blob
...
repository: Better error message if blob is larger than 4GB
2024-04-22 21:45:06 +02:00
Michael Eischer
20d8eed400
repository: streamPack: separate requests for gap larger than 1MB
...
With most cloud providers, traffic is much more expensive than API
calls. Thus slightly bias streamPack towards a bit more API calls in
exchange for slightly less traffic.
2024-04-22 21:21:23 +02:00
Michael Eischer
cf700d8794
repository: streamPack: reuse zstd decoder
2024-04-22 21:21:23 +02:00
Michael Eischer
666a0b0bdb
repository: streamPack: replace streaming with chunked download
...
Due to the interface of streamPack, we cannot guarantee that operations
progress fast enough that the underlying connections remains open. This
introduces partial failures which massively complicate the error
handling.
Switch to a simpler approach that retrieves the pack in chunks of 32MB.
If a blob is larger than this limit, then it is downloaded separately.
To avoid multiple copies in memory, an auxiliary interface
`discardReader` is introduced that allows directly accessing the
downloaded byte slices, while still supporting the streaming used by the
`check` command.
2024-04-22 21:21:23 +02:00
Michael Eischer
621012dac0
repository: Add blob loading fallback to LoadBlobsFromPack
...
Try to retrieve individual blobs via LoadBlob if streaming did not work.
2024-04-21 21:35:55 +02:00
Michael Eischer
6c6dceade3
global: unify backend open and create
2024-04-19 22:26:14 +02:00
Michael Eischer
10355c3fb6
repository: Better error message if blob is larger than 4GB
2024-04-19 22:00:35 +02:00
Michael Eischer
228b35f074
Merge pull request #4769 from will-ca/patch-1
...
Tiny wording clarification in `restic-stats.1`.
2024-04-18 19:00:39 +00:00
will-ca
6aced61c72
Tiny docs wording clarification.
2024-04-18 07:29:55 +00:00
Michael Eischer
4d22412e0c
Merge pull request #4766 from coderwander/master
...
Fix struct names
2024-04-18 06:18:19 +00:00
coderwander
a82ed71de7
Fix struct names
...
Signed-off-by: coderwander <770732124@qq.com>
2024-04-18 10:02:09 +08:00
Michael Eischer
2173c69280
Merge pull request #4770 from testwill/close_files
...
fix: close files
2024-04-17 16:50:20 +00:00
Michael Eischer
001bb71676
repair packs: Properly close backup files
2024-04-17 18:32:30 +02:00
Michael Eischer
c9191ea72c
forget: cleanup verbose output on snapshot deletion error
2024-04-14 14:17:40 +02:00
Michael Eischer
09587e6c08
repository: duplicate a few blobs in prune tests
2024-04-14 13:57:19 +02:00
Michael Eischer
defd7ae729
prune/repair index: reset in-memory index after command
...
The current in-memory index becomes stale after prune or repair index
have run. Thus, just drop the in-memory index altogether once these
commands have finished.
2024-04-14 13:46:24 +02:00
Michael Eischer
038586dc9d
repository: add minimal test for prune
2024-04-14 13:45:17 +02:00
Michael Eischer
d8622c86eb
prune: clean up internal interface
2024-04-14 13:45:15 +02:00
Michael Eischer
8d507c1372
repository: add basic test for RepairIndex
2024-04-14 13:45:15 +02:00
Michael Eischer
310db03c0e
repair index: improve log output if index cannot be deleted
...
The operation will always fail with an error if an index cannot be
deleted. Thus, this change is purely cosmetic.
2024-04-14 13:45:13 +02:00
Michael Eischer
7d1b9cde34
repository: use normal Init method in tests
2024-04-14 13:45:11 +02:00
Michael Eischer
b25fc2c89d
repository: remove redundant flushes from tests
2024-04-14 13:45:10 +02:00
Michael Eischer
c65459cd8a
repository: speed up tests
2024-04-14 13:45:10 +02:00
Michael Eischer
eda9f7beb4
ui/progress: add helper to print messages during tests
2024-04-14 13:45:08 +02:00
Michael Eischer
35277b7797
backend/mem: cleanup not found error message
2024-04-14 13:45:06 +02:00
Michael Eischer
7ba5e95a82
check: allow tests to only verify pack&index integrity
2024-04-14 13:45:04 +02:00
Michael Eischer
4c9a10ca37
repair packs: deduplicate index rebuild
2024-04-14 13:45:02 +02:00
Michael Eischer
85e4021619
prune: move additional option checks to repository
2024-04-14 13:44:58 +02:00
Michael Eischer
55d56db31b
Merge pull request #4743 from MichaelEischer/deprecate-s3legacy-layout
...
Deprecate s3legacy layout
2024-04-11 22:09:34 +02:00
Michael Eischer
fc3b548625
prune: move logic into repository package
2024-04-10 21:30:52 +02:00
Michael Eischer
df9d4b455d
prune: prepare for moving code to repository package
2024-04-10 21:30:52 +02:00
Michael Eischer
866ddf5698
repair index: refactor code into repository package
2024-04-10 21:30:52 +02:00
Michael Eischer
32a234b67e
prune/forget/repair index: convert output to use progress.Printer
2024-04-10 21:30:52 +02:00
Michael Eischer
739d11c2eb
forget: replace usage of DeleteFilesChecked
...
This simplifies refactoring prune into the repository package.
2024-04-10 21:30:52 +02:00
Michael Eischer
591b421c4a
Deprecate s3legacy layout
2024-04-10 21:27:56 +02:00
Michael Eischer
8efc3a8b7d
Merge pull request #4668 from MichaelEischer/backup-xattr-parent-enoperm
...
backup: Ignore xattr.list permission error for parent directories
2024-04-10 21:25:28 +02:00
Michael Eischer
bf054c09d2
backup: Ignore xattr.list permission error for parent directories
...
On FreeBSD, limited users may not be able to even list xattrs for the
parent directories above the snapshot source paths. As this can cause
the backup to fail, just ignore those errors.
2024-04-10 20:46:15 +02:00
Michael Eischer
0747cf5319
cache: ignore ErrNotExist during cleanup of old files
...
Two restic processes running concurrently can try to remove the same
files from the cache. This could cause one process to fail with an error
if the other one has already remove a file that the current process also
tries to delete.
2024-04-10 19:25:51 +02:00
Michael Eischer
6091029fd6
Merge pull request #4756 from mgeisler/patch-1
...
doc: fix typo in 047_tuning_backup_parameters.rst
2024-04-07 19:24:09 +00:00
Martin Geisler
09d2183351
doc: fix typo in 047_tuning_backup_parameters.rst
2024-04-07 18:05:53 +02:00
Michael Eischer
a4b7ebecfc
Merge pull request #4750 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.40.0
...
build(deps): bump cloud.google.com/go/storage from 1.39.0 to 1.40.0
2024-04-03 20:10:41 +00:00
dependabot[bot]
ba136b31b8
build(deps): bump cloud.google.com/go/storage from 1.39.0 to 1.40.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.39.0 to 1.40.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.39.0...spanner/v1.40.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-04-03 20:03:22 +00:00
Michael Eischer
f328111a6e
Merge pull request #4751 from restic/dependabot/go_modules/golang.org/x/net-0.23.0
...
build(deps): bump golang.org/x/net from 0.21.0 to 0.23.0
2024-04-03 19:52:29 +00:00
Michael Eischer
9fb017e67a
Merge pull request #4745 from MichaelEischer/full-id-key-list
...
key list: include full key id in JSON output
2024-04-03 21:50:02 +02:00
Michael Eischer
49f98f25fc
Merge pull request #4742 from MichaelEischer/consistent-rtest-import
...
Use consistent alias for interal/test package
2024-04-03 21:47:32 +02:00
dependabot[bot]
96c602a6de
build(deps): bump golang.org/x/net from 0.21.0 to 0.23.0
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.21.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.21.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-04-03 19:41:38 +00:00
Michael Eischer
1d0a20dd58
Merge pull request #4748 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.10.0
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.9.2 to 1.10.0
2024-04-03 19:34:47 +00:00
Michael Eischer
6cca1d5705
Merge pull request #4655 from ae-govau/unixsocket
...
Enhancement: option to send HTTP over unix socket
2024-04-03 19:29:21 +00:00
dependabot[bot]
f8a72ac2a3
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.9.2 to 1.10.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.9.2...sdk/azcore/v1.10.0 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-04-01 02:32:19 +00:00
Michael Eischer
5145c8f9c0
key list: include full key id in JSON output
2024-03-31 12:25:20 +02:00
Michael Eischer
831fc4413d
Merge pull request #4737 from stephan0307/3117
...
json output forget command: added id's in snapshots within reasons object
2024-03-29 13:11:24 +00:00
Stephan Paul
df07814ec2
forget json output: added id's in snapshots within reasons object
...
In order to evaluate the keep reasons for snapshots, there should be also the id's
to compare it with snapshots within the keep object. (See also Issue #3117 )
In order to avoid output parameters also changed function addJSONSnapshots to asJSONSnapshots
2024-03-29 11:30:00 +01:00
Michael Eischer
ec2b79834a
use consistent alias for interal/test package
2024-03-29 00:24:03 +01:00
Michael Eischer
510f6f06b0
Merge pull request #4709 from MichaelEischer/refactor-locking
...
Refactor locking into repository package
2024-03-28 23:53:09 +01:00
Michael Eischer
07eb6c315b
add changelog for locking refactor
2024-03-28 23:46:58 +01:00
Michael Eischer
5e98f1e2eb
repository: fix test setup race conditions
2024-03-28 23:17:02 +01:00
Michael Eischer
8155dbe711
correctly lock repository in integration tests
2024-03-28 23:17:02 +01:00
Michael Eischer
d18726cd70
ls: add missing read lock
...
As `ls` reads data from the repository, it must acquire a read lock
unless `--no-lock` was specified. The old behavior is equivalent to `ls
--no-lock`.
2024-03-28 23:17:02 +01:00
Michael Eischer
dc441c57a7
repository: unify repository initialization in tests
...
Tests should use a helper from internal/repository/testing.go to
construct a Repository object.
2024-03-28 23:17:02 +01:00
Michael Eischer
3ba1fa3cee
repository: remove a few global variables
2024-03-28 23:17:02 +01:00
Michael Eischer
044e8bf821
repository: parallelize lock tests
2024-03-28 23:17:02 +01:00
Michael Eischer
e8df50fa3c
repository: remove global list of locks
2024-03-28 22:46:33 +01:00
Michael Eischer
cbb5f89252
lock: move code to repository package
2024-03-28 22:46:33 +01:00
Michael Eischer
118a69a84b
lock: replace lockRepo(Exclusive) with openWith(Read/Write/Exclusive)Lock
...
The new functions much better convey the intent behind the lock
request. This allows cleanly integrating noLock (for read) and dryRun
(write/exclusive) handling.
There are only minor changes to existing behavior with two exceptions:
- `tag` no longer accepts the `--no-lock` flag. As it replaces files in
the repository, this always requires an exclusive lock.
- `debug examine` now returns an error if both `--extract-pack` and
`--no-lock` are given.
2024-03-28 22:46:33 +01:00
Michael Eischer
7f9ad1c3db
Merge pull request #4705 from MichaelEischer/snapshot-statistics
...
Store snapshot statistics & print snapshot size
2024-03-28 22:41:45 +01:00
Michael Eischer
71b6284155
Merge pull request #4006 from MichaelEischer/deviceID-only-for-hardlinks
...
archiver: only store deviceID for hardlinks
2024-03-28 22:33:28 +01:00
Michael Eischer
cf81f8ced6
stats: only check for hardlinks for files with more than one link
2024-03-28 21:29:27 +01:00
Michael Eischer
21cf38fe96
add changelog for deviceID only for hardlinks
2024-03-28 19:32:50 +01:00
Michael Eischer
d705741571
backup: test that deviceID is only stored for hardlinks
2024-03-28 19:32:50 +01:00
Michael Eischer
a9b3d86c4f
features: remove example feature
2024-03-28 19:12:07 +01:00
Michael Eischer
a26d6ffa72
archiver: move deviceID handling behind feature flag
2024-03-28 19:12:07 +01:00
Michael Eischer
2ba21fe72b
archiver: only store deviceID for hardlinks
...
The deviceID can change e.g. when backing up from filesystem snapshot.
It is only used for hardlink detection. Thus there it is not necessary
to store it for everything else.
2024-03-28 19:12:07 +01:00
Michael Eischer
870904d3ae
Merge pull request #4731 from facutuesca/powershell-completion-doc
...
doc: Add instructions to configure PowerShell completions
2024-03-28 17:46:55 +00:00
Facundo Tuesca
15555c9898
doc: Use consistent case for PROFILE env variable in PowerShell
2024-03-28 18:35:18 +01:00
Michael Eischer
63a2350c9e
Merge pull request #4741 from MichaelEischer/ci-upgrade-golangci-lint
...
CI: Update golangci-lint to version 1.57.1
2024-03-28 18:27:05 +01:00
Michael Eischer
8876e3025b
Merge pull request #4724 from MichaelEischer/disable-old-index
...
Deprecated legacy index format
2024-03-28 18:24:47 +01:00
Michael Eischer
4f4979f4e8
Merge pull request #4740 from MichaelEischer/ci-upgrade-docker-actions
...
CI: update docker actions
2024-03-28 18:16:38 +01:00
Michael Eischer
1497525e15
CI: Update golangci-lint to version 1.57.1
2024-03-28 18:15:50 +01:00
Michael Eischer
a8face3a25
Merge pull request #4739 from MichaelEischer/ci-allow-annotations
...
CI: Allow golangci-lint to annotate PRs
2024-03-28 18:02:16 +01:00
Michael Eischer
aee6d311f1
CI: update docker actions
2024-03-28 18:01:48 +01:00
Michael Eischer
6ac7519188
add changelog for rest unix socket support
2024-03-28 17:41:41 +01:00
Michael Eischer
add37fcd9f
CI: uses rest-server from master branch until unix sockets are released
2024-03-28 17:41:41 +01:00
Adam Eijdenberg
6e775d3787
Enhancement: option to send HTTP over unix socket
...
add tests for unix socket connection
switch HTTP rest-server test to use any free port
allow rest-server test graceful shutdown opportunity
2024-03-28 17:41:41 +01:00
Michael Eischer
5c4a4b4a30
CI: Allow golangci-lint to annotate PRs
2024-03-28 17:09:59 +01:00
Michael Eischer
d1d773cfcd
Merge pull request #4727 from avoidalone/master
...
fix some typos
2024-03-17 18:52:40 +00:00
Facundo Tuesca
521713fc94
doc: Add instructions to configure PowerShell completions
2024-03-16 18:54:27 +01:00
avoidalone
ac948fccda
fix some typos
...
Signed-off-by: avoidalone <wuguangdong@outlook.com>
2024-03-11 14:35:12 +08:00
rawtaz
9284f7413a
Merge pull request #4725 from leoheitmannruiz/master
...
Minor README.md cleanups
2024-03-10 12:24:07 +00:00
rawtaz
1287b977b4
Merge pull request #4726 from leoheitmannruiz/patch-1
...
Capitalize Homebrew and minor edit in wording
2024-03-10 12:23:48 +00:00
Leo Heitmann Ruiz
00f762373f
Capitalize Homebrew
2024-03-10 00:20:26 +01:00
Leo Heitmann Ruiz
9f3e1462c0
Minor README.md cleanups
2024-03-09 23:56:16 +01:00
Michael Eischer
69ca12d2eb
check: treat legacy index format as errors
2024-03-09 18:36:33 +01:00
Michael Eischer
98a6817d01
add changelog for legacy index deprecation
2024-03-09 18:35:00 +01:00
Michael Eischer
f8852f0eb6
repair index: fix deletion of legacy indexes
2024-03-09 18:21:22 +01:00
Michael Eischer
1a8bf358f1
index: deprecate legacy index format
2024-03-09 18:21:14 +01:00
Michael Eischer
396a61a992
Merge pull request #4666 from MichaelEischer/feature-flags
...
Implement feature flags
2024-03-09 17:36:29 +01:00
Michael Eischer
a9b64cd7ad
features: print warning for stable/depreacted feature flags
2024-03-09 17:29:52 +01:00
Michael Eischer
fe68d2cafb
add feature flag documentation
2024-03-09 17:29:52 +01:00
Michael Eischer
70839155f2
features: add tests
2024-03-09 17:29:52 +01:00
Michael Eischer
1c77c51a03
features: initialize based on RESTIC_FEATURES environment variable
2024-03-09 17:29:52 +01:00
Michael Eischer
5974a79497
features: add basic feature flag implementation
2024-03-09 17:29:52 +01:00
Michael Eischer
0589da60b3
Merge pull request #4717 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.39.0
...
build(deps): bump cloud.google.com/go/storage from 1.37.0 to 1.39.0
2024-03-07 14:10:24 +00:00
dependabot[bot]
608116817b
build(deps): bump cloud.google.com/go/storage from 1.37.0 to 1.39.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.37.0 to 1.39.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/spanner/v1.37.0...spanner/v1.39.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-07 13:59:23 +00:00
Michael Eischer
f742da8b2f
Merge pull request #4722 from konidev20/upgrade-docker-golang-base-image-to-1-22
...
docker: update the base image to golang:1.22-alpine
2024-03-07 13:44:48 +00:00
Michael Eischer
af1684743f
Merge pull request #4716 from restic/dependabot/go_modules/github.com/klauspost/compress-1.17.7
...
build(deps): bump github.com/klauspost/compress from 1.17.6 to 1.17.7
2024-03-07 13:41:46 +00:00
Michael Eischer
5b9de4d8b7
Merge pull request #4715 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob-1.3.1
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 1.2.1 to 1.3.1
2024-03-07 13:40:44 +00:00
Michael Eischer
fceef67abe
Merge pull request #4718 from restic/dependabot/go_modules/github.com/spf13/cobra-1.8.0
...
build(deps): bump github.com/spf13/cobra from 1.7.0 to 1.8.0
2024-03-07 13:37:17 +00:00
Michael Eischer
8506cae710
Merge pull request #4714 from restic/dependabot/github_actions/golangci/golangci-lint-action-4
...
build(deps): bump golangci/golangci-lint-action from 3 to 4
2024-03-07 13:34:21 +00:00
Michael Eischer
87d47ef189
Merge pull request #4713 from restic/dependabot/github_actions/docker/login-action-5139682d94efc37792e6b54386b5b470a68a4737
...
build(deps): bump docker/login-action from 3d58c274f17dffee475a5520cbe67f0a882c4dbb to 5139682d94efc37792e6b54386b5b470a68a4737
2024-03-07 13:33:49 +00:00
Michael Eischer
55abf25ea8
Merge pull request #4719 from restic/dependabot/go_modules/golang.org/x/oauth2-0.17.0
...
build(deps): bump golang.org/x/oauth2 from 0.16.0 to 0.17.0
2024-03-07 13:32:36 +00:00
Srigovind Nayak
b48b1fa2c9
docker: update the base image to golang:1.22-alpine
2024-03-03 16:18:13 +05:30
dependabot[bot]
8e7f29ae28
build(deps): bump golang.org/x/oauth2 from 0.16.0 to 0.17.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.16.0 to 0.17.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.16.0...v0.17.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:25:47 +00:00
dependabot[bot]
79e8ddac3f
build(deps): bump github.com/spf13/cobra from 1.7.0 to 1.8.0
...
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra ) from 1.7.0 to 1.8.0.
- [Release notes](https://github.com/spf13/cobra/releases )
- [Commits](https://github.com/spf13/cobra/compare/v1.7.0...v1.8.0 )
---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:25:41 +00:00
dependabot[bot]
b5a9b5d0bc
build(deps): bump github.com/klauspost/compress from 1.17.6 to 1.17.7
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.17.6 to 1.17.7.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.17.6...v1.17.7 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:25:00 +00:00
dependabot[bot]
f185c80cf0
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go ) from 1.2.1 to 1.3.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azidentity/v1.2.1...sdk/azcore/v1.3.1 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:24:46 +00:00
dependabot[bot]
70c8aaa303
build(deps): bump golangci/golangci-lint-action from 3 to 4
...
Bumps [golangci/golangci-lint-action](https://github.com/golangci/golangci-lint-action ) from 3 to 4.
- [Release notes](https://github.com/golangci/golangci-lint-action/releases )
- [Commits](https://github.com/golangci/golangci-lint-action/compare/v3...v4 )
---
updated-dependencies:
- dependency-name: golangci/golangci-lint-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:17:27 +00:00
dependabot[bot]
e1a588b75c
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from 3d58c274f17dffee475a5520cbe67f0a882c4dbb to 5139682d94efc37792e6b54386b5b470a68a4737.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](3d58c274f1...5139682d94
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-03-01 01:17:20 +00:00
Michael Eischer
e71660cd1e
backup: rename data_added_in_repo statistic to data_added_packed
2024-02-25 20:40:52 +01:00
Aneesh Nireshwalia
062d408987
Clean up SecurityDescriptor helper
2024-02-24 14:23:04 -07:00
Aneesh Nireshwalia
5764300022
Add changelog and fix lint error
2024-02-24 13:47:49 -07:00
Aneesh Nireshwalia
c0a1b9ada5
Update docs for security descriptors
2024-02-24 13:28:18 -07:00
Aneesh Nireshwalia
90916f53de
Add test cases for security descriptors
2024-02-24 13:27:01 -07:00
Aneesh Nireshwalia
70cf8e3788
Add support for backup/restore of security descriptors
2024-02-24 13:25:28 -07:00
Aneesh Nireshwalia
e3e59fef24
Fix CombineErrors and fillExtendedAttr error handling
2024-02-24 13:22:34 -07:00
Aneesh Nireshwalia
09ce1b4e58
Create helper for SecurityDescriptor related functions
2024-02-24 13:16:25 -07:00
Michael Eischer
6a13e451b1
document snapshot statistics
2024-02-23 22:32:04 +01:00
Michael Eischer
a8f5684f68
archiver: test statistics in snapshot
2024-02-23 22:05:15 +01:00
Michael Eischer
681395955e
archiver: test backup summary calculation
2024-02-23 21:46:39 +01:00
Michael Eischer
b6520038fd
snapshots: Print snapshot size stored in snapshots
2024-02-23 20:27:13 +01:00
Michael Eischer
38f91d3b5e
backup: store statistics in snapshot
2024-02-23 20:27:13 +01:00
Michael Eischer
86897314d5
backup: expose data_added_in_repo in JSON output
...
The value describes how much data was added after compression.
Previously, it was only available in the text output.
2024-02-23 20:27:13 +01:00
Michael Eischer
a59f654fa6
archiver: refactor summary collection from ui into the archiver
2024-02-23 20:27:13 +01:00
Michael Eischer
8b1a85711f
archiver: unexport save/saveDir/saveTree methods
2024-02-23 20:24:21 +01:00
Michael Eischer
b953dc8f58
Merge pull request #4611 from zmanda/windows-metadata-support
...
Back up and restore windows metadata like created ts, file attribs like hidden, readonly, encrypted with a common extensible mechanism
2024-02-23 18:16:09 +00:00
Aneesh Nireshwalia
e8211cb64a
Add changelog and update docs for windows attr
2024-02-22 17:59:56 -07:00
Aneesh Nireshwalia
4bbd25a37f
Add tests for generic attribute changes
2024-02-22 17:55:50 -07:00
Aneesh Nireshwalia
d4be734c73
Handle readonly empty files in windows
2024-02-22 17:54:43 -07:00
Aneesh Nireshwalia
eeb1aa5388
Add ability to report warnings to terminal
...
Report warnings to terminal when unrecognized generic attributes are found in the repository.
2024-02-22 17:52:26 -07:00
Aneesh Nireshwalia
0962917974
Support windows metadata using generic attribs
...
Add new generic_attributes attribute in Node.
Use the generic attributes to add support for creation time and file attributes like hidden, readonly, encrypted in windows. Handle permission errors for readonly files in windows.
Handle backup and restore of encrypted attributes using windows system calls.
2024-02-22 17:31:20 -07:00
Aneesh Nireshwalia
62a8a599f1
Add optional messages for Equals helper
2024-02-22 16:58:12 -07:00
Aneesh Nireshwalia
94de87d4b7
Add CombineErrors helper function
2024-02-22 16:57:00 -07:00
Michael Eischer
c6311c1e32
Merge pull request #4703 from ferringb/master
...
Catch SIGTERM, run cleanup
2024-02-22 21:06:29 +00:00
Michael Eischer
0a65a0f94f
update comment
2024-02-22 22:00:42 +01:00
Brian Harring
b41107dcaf
Add changelog for SIGTERM bugfix.
...
Signed-off-by: Brian Harring <ferringb@gmail.com>
2024-02-19 11:31:48 +01:00
Brian Harring
30e979d252
Catch SIGTERM, run cleanup
...
The previous code only ran cleanup (lock release for example) on SIGINT. For
anyone running restic in a container, the signal is going to be SIGTERM which
means containerized execution would leave locks behind.
While this could be addressed via interposing dumb-init to translate the signal,
a `kill` invocation is going to default to SIGTERM, so the same problem exists
for non container users.
Signed-off-by: Brian Harring <ferringb@gmail.com>
2024-02-19 11:12:15 +01:00
Michael Eischer
cfbeb2cde5
Merge pull request #4701 from MichaelEischer/better-streampack-errors
...
repository: Improve StreamPack error messages
2024-02-18 16:57:56 +01:00
Michael Eischer
80754dbf0c
Merge pull request #4664 from MichaelEischer/ls-unified-json-output
...
ls: include standard `message_type` field in output
2024-02-18 15:47:41 +00:00
Michael Eischer
4c3218ef9f
repository: include packID in StreamPack for decrypt/decompress errors
2024-02-17 19:38:01 +01:00
Michael Eischer
18b0bbbf42
repository: use fmt.Errorf in StreamPacks
2024-02-17 19:37:32 +01:00
Michael Eischer
6fbb470835
Merge pull request #4665 from MichaelEischer/check-repair-packs
...
check: Suggest usage of `repair packs` if pack files are damaged
2024-02-17 15:54:07 +00:00
Michael Eischer
0a36d193d8
add changelog for enhanced repair packs
2024-02-12 21:43:35 +01:00
Michael Eischer
69304cd74f
check: clarify repair pack usage
2024-02-12 21:43:35 +01:00
Michael Eischer
c3b0e6d004
Merge pull request #4700 from MichaelEischer/remove-duplicate-changelog-entries
...
remove changelogs that are already included in restic 0.16.4
2024-02-12 21:40:04 +01:00
Michael Eischer
9e3703ded5
remove changelogs that are already included in restic 0.16.4
2024-02-12 20:39:31 +01:00
Michael Eischer
527a3ff2b2
check: link to troubleshooting guide
2024-02-12 20:25:15 +01:00
Michael Eischer
ed4a4f8748
check: exclude inaccessible files from the repair pack suggestion
2024-02-12 20:25:15 +01:00
Michael Eischer
4073299a7c
check: fix missing error if blob is invalid
2024-02-12 20:20:13 +01:00
Michael Eischer
6397615fbb
check: document that check will show repair pack instructions
2024-02-12 20:20:13 +01:00
Michael Eischer
544fe38786
check: suggest repair pack for all damaged packs
2024-02-12 20:20:13 +01:00
Michael Eischer
772e3416d1
repair pack: drop feature flag
2024-02-12 20:20:12 +01:00
Michael Eischer
22a3cea1b3
checker: wrap all pack errors in ErrPackData
2024-02-12 20:19:32 +01:00
Michael Eischer
19bf2cf52d
Merge pull request #4697 from MichaelEischer/report-blob-errors
...
backup: report files whose chunks failed to upload
2024-02-12 20:18:51 +01:00
Michael Eischer
5b5d506472
backup: report files whose chunks failed to upload
2024-02-11 22:43:26 +01:00
Michael Eischer
dde556e8e8
Merge pull request #4696 from MichaelEischer/fix-exclude-load-error-msg
...
backup: improve error message if exclude file cannot be loaded
2024-02-11 21:35:31 +00:00
Michael Eischer
ee1ff3c1d0
backup: improve error message if exclude file cannot be loaded
2024-02-11 22:26:13 +01:00
Michael Eischer
667a2f5369
Merge pull request #4694 from restic/update-go-versions
...
Update go versions
2024-02-11 17:10:12 +00:00
Michael Eischer
2ab18a92e6
CI: keep tests for Go 1.19
2024-02-10 23:42:34 +01:00
Alexander Neumann
c0514dd8ba
Fix linter errors (except for tests)
2024-02-10 22:58:10 +01:00
Alexander Neumann
a8cda0119c
Upgrade golangci-lint
2024-02-10 22:08:43 +01:00
Alexander Neumann
9720935c56
Update Go version for tests to 1.22
2024-02-10 21:56:01 +01:00
Michael Eischer
68cc327b15
Merge pull request #4692 from 27149chen/dump-to-existing-file
...
feat: dump flag --target should be allowed to write existing file
2024-02-10 17:44:54 +00:00
Michael Eischer
15d6fa1f83
dump: update docs for --target option
2024-02-10 18:39:06 +01:00
lou
80db02fc35
dump flag --target should be allowed to write existing file
...
Signed-off-by: lou <alex1988@outlook.com>
2024-02-10 18:39:06 +01:00
Michael Eischer
6a2b10e2a8
Merge pull request #4685 from konidev20/fix-gh-4676-sub-commands-for-key-management
...
Move key add, list, remove and passwd as separate sub-commands and improve key sub-command documentation
2024-02-08 19:59:16 +00:00
Michael Eischer
e46b21ab80
key: fix integration test for invalid arguments
2024-02-08 20:52:30 +01:00
Michael Eischer
eb389a2d25
Merge pull request #4687 from MichaelEischer/upgrade-zstd-library
...
Upgrade zstd library to latest version
2024-02-08 20:29:13 +01:00
Srigovind Nayak
795d33b3ee
key: move add, list, remove, passwd to sub-commands
...
docs: improve the sub-command docs
changelog: add the unreleased changelog for the key command updates
key: update integration tests
2024-02-06 01:47:43 +05:30
Michael Eischer
0cffdb7493
Merge pull request #4682 from konidev20/feat-add-target-for-dump-command
...
feat: set --target for dump command
2024-02-05 19:16:42 +00:00
Michael Eischer
f5ffa40652
dump: minor cleanups
2024-02-05 20:10:52 +01:00
Srigovind Nayak
175c14b5c9
dump: add --target option
2024-02-05 20:10:52 +01:00
Michael Eischer
bca099ac7f
Upgrade zstd library to latest version
...
The data corruption bug is fixed, thus remove the override.
2024-02-05 19:53:02 +01:00
Michael Eischer
0f09a8870c
Merge pull request #4684 from konidev20/fix-gh-4658-update-azure-documentation
...
Update azure storage account authentication documentation
2024-02-04 20:13:18 +00:00
Srigovind Nayak
5771c4ecfb
docs: update environment variables for `az login` to azure backend
2024-02-05 01:32:43 +05:30
Michael Eischer
b63bfd2257
Merge branch 'patch-release'
2024-02-04 20:21:42 +01:00
Alexander Neumann
0f9fa44de5
Set development version for 0.16.4
2024-02-04 19:50:56 +01:00
Alexander Neumann
3786536dc1
Add version for 0.16.4
2024-02-04 19:50:52 +01:00
Alexander Neumann
811be5984d
Update manpages and auto-completion
2024-02-04 19:50:51 +01:00
Alexander Neumann
b0ead75de5
Generate CHANGELOG.md for 0.16.4
2024-02-04 19:50:34 +01:00
Alexander Neumann
6cd2804bff
Prepare changelog for 0.16.4
2024-02-04 19:50:34 +01:00
Michael Eischer
a72c2b74f3
Apply changelog entry / documentation improvements from review
2024-02-04 19:10:06 +01:00
Michael Eischer
261b1455c7
add documentation for --no-extra-verify option
2024-02-04 19:10:06 +01:00
Michael Eischer
2a0bd2b637
rename `--no-verify-pack` to `--no-extra-verify`
2024-02-04 19:10:05 +01:00
Michael Eischer
4589da7eb9
add data verification changelog entry
2024-02-04 19:09:49 +01:00
Michael Eischer
75e72d826c
pack: verify integrity of pack file header
2024-02-04 19:09:49 +01:00
Michael Eischer
d8916bc3d9
repository: ask users to report corrupted data while saving blobs
2024-02-04 19:09:49 +01:00
Michael Eischer
dc11d012bb
Make --no-verify-pack globally available
...
Verifying all blobs before upload comes with a notable performance
impact. Allow users to skip it if necessary.
2024-02-04 19:09:49 +01:00
Michael Eischer
8ef5425351
repository: test verification of blobs/unpacked data
2024-02-04 19:09:46 +01:00
Michael Eischer
885431ec2b
repository: Allow skipping verification for tests
...
Some tests have to explicitly create pack files with blobs that don't
match their ID. For those blobs the builtin verification of the
repository must be disabled.
2024-02-04 19:08:30 +01:00
Michael Eischer
cb85fb46dd
backup: verify unpacked files before upload
2024-02-04 19:07:48 +01:00
Michael Eischer
2f30c940b2
backup: verify blobs before upload
...
This only covers the blobs themselves, the pack header is not verified
so far. Unpacked files are also not covered by the integrity check.
2024-02-04 19:07:48 +01:00
Michael Eischer
0ea62b5ac6
repository: make repo.Options configurable for test repos
2024-02-04 19:07:46 +01:00
Michael Eischer
29e1caf825
add changelog draft for data corruption on max compression
2024-02-04 19:05:51 +01:00
Michael Eischer
0164f5310d
Downgrade klauspost/compress to fix data corruption at max. compression
2024-02-04 19:05:50 +01:00
Michael Eischer
d5e662315a
Merge pull request #4681 from MichaelEischer/verify-integrity-on-upload
...
backup: verify blobs before upload
2024-02-04 18:04:27 +00:00
Michael Eischer
effe76aaf5
Merge pull request #4679 from MichaelEischer/workaround-compression-bug
...
Downgrade klauspost/compress to fix data corruption at max. compression
2024-02-04 18:03:34 +00:00
Michael Eischer
5957417b1f
Apply changelog entry / documentation improvements from review
2024-02-04 18:55:41 +01:00
Michael Eischer
219d8e3c18
add changelog draft for data corruption on max compression
2024-02-04 18:11:48 +01:00
Michael Eischer
a737fe1e47
add documentation for --no-extra-verify option
2024-02-04 17:11:49 +01:00
Michael Eischer
86b38a0b17
rename `--no-verify-pack` to `--no-extra-verify`
2024-02-04 17:01:05 +01:00
Michael Eischer
7d31180fe6
add data verification changelog entry
2024-02-04 15:48:11 +01:00
Michael Eischer
c32e5e2abb
pack: verify integrity of pack file header
2024-02-04 15:31:42 +01:00
Michael Eischer
c97a271e89
repository: ask users to report corrupted data while saving blobs
2024-02-04 15:31:42 +01:00
Michael Eischer
66e8971659
Make --no-verify-pack globally available
...
Verifying all blobs before upload comes with a notable performance
impact. Allow users to skip it if necessary.
2024-02-04 15:31:42 +01:00
Michael Eischer
193140525c
repository: test verification of blobs/unpacked data
2024-02-04 15:31:42 +01:00
Michael Eischer
96518d7c4a
Merge pull request #4674 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.37.0
...
build(deps): bump cloud.google.com/go/storage from 1.34.0 to 1.37.0
2024-02-03 17:23:49 +00:00
Michael Eischer
2dbb18128c
repository: Allow skipping verification for tests
...
Some tests have to explicitly create pack files with blobs that don't
match their ID. For those blobs the builtin verification of the
repository must be disabled.
2024-02-03 18:22:47 +01:00
Michael Eischer
30a84e9003
backup: verify unpacked files before upload
2024-02-03 18:22:47 +01:00
Michael Eischer
c01a0c6da7
backup: verify blobs before upload
...
This only covers the blobs themselves, the pack header is not verified
so far. Unpacked files are also not covered by the integrity check.
2024-02-03 18:22:47 +01:00
Michael Eischer
16e3f79e8b
repository: make repo.Options configurable for test repos
2024-02-03 18:22:47 +01:00
Michael Eischer
bb92b487f7
repository: fix repack test
2024-02-03 18:22:47 +01:00
dependabot[bot]
cf7cad11de
build(deps): bump cloud.google.com/go/storage from 1.34.0 to 1.37.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.34.0 to 1.37.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.34.0...spanner/v1.37.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-02-02 20:40:57 +00:00
Michael Eischer
370d9c31f4
Merge pull request #4671 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob-1.2.1
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 1.2.0 to 1.2.1
2024-02-02 20:31:16 +00:00
Michael Eischer
6581133e85
Merge pull request #4675 from restic/dependabot/go_modules/golang.org/x/oauth2-0.16.0
...
build(deps): bump golang.org/x/oauth2 from 0.15.0 to 0.16.0
2024-02-02 20:29:39 +00:00
Michael Eischer
207a4a5e8e
Downgrade klauspost/compress to fix data corruption at max. compression
2024-02-02 20:10:29 +01:00
Michael Eischer
cbf9cd4a7f
Merge pull request #4670 from joram-berger/patch-1
...
Link to Go Match syntax directly in 040_backup.rst
2024-02-02 18:40:13 +00:00
dependabot[bot]
552f01662b
build(deps): bump golang.org/x/oauth2 from 0.15.0 to 0.16.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.15.0 to 0.16.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.15.0...v0.16.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-02-01 01:53:12 +00:00
dependabot[bot]
7f5ea511bc
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go ) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/v1.2...sdk/azidentity/v1.2.1 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-02-01 01:51:32 +00:00
Joram Berger
b07afa9b02
Link to Go Match syntax directly in 040_backup.rst
...
The docs linked to filepath.Glob (which is used) but the syntax is described in the documentation of filepath.Match. So it makes sense to link that directly.
2024-01-31 23:40:21 +01:00
Michael Eischer
8b08b522c9
Merge pull request #4669 from MichaelEischer/fix-rewrite-typo
...
rewrite: fix typo in dry-run output
2024-01-31 22:08:12 +01:00
Michael Eischer
eaf9659efc
Merge pull request #4657 from numerigraphe/fix-key-add-id
...
Properly report the ID of newly added keys
2024-01-31 20:59:41 +00:00
Michael Eischer
ba136ff60c
rewrite: fix typo in dry-run output
2024-01-31 21:48:37 +01:00
Lionel Sausin
8fbe328371
Properly report the ID of newly added keys
...
Other commands like key list and key remove show the key's ID.
Showing the ID here lets users easily reuse the ID as a key hint for subsequent
commands.
In particular, a key hint is needed when the repository has many keys - otherwise
opening the repository may fail with "Fatal: maximum number of keys reached" even
when a proper password is provided.
Fixes #4656
2024-01-29 10:12:49 +01:00
Michael Eischer
4273e06a43
Merge pull request #4662 from MichaelEischer/clarify-backup-ignore-inode
...
backup: clarify that --ignore-inode also ignores ctime
2024-01-27 18:25:11 +01:00
Michael Eischer
248c144f72
Merge pull request #4663 from MichaelEischer/key-subcommand-bugfix
...
key: return an error if subcommand is unknown
2024-01-27 18:24:50 +01:00
Michael Eischer
5dca8a70d5
ls: include standard `message_type` field in output
2024-01-27 15:48:24 +01:00
Michael Eischer
765729d009
key: return an error if subcommand is unknown
2024-01-27 15:33:49 +01:00
Michael Eischer
a09d51d96c
backup: clarify that --ignore-inode also ignores ctime
2024-01-27 13:42:29 +01:00
Michael Eischer
e44e4b00a6
Merge pull request #4550 from ndecker/ls-ncdu
...
Ls ncdu
2024-01-27 12:27:35 +00:00
Michael Eischer
10e71af759
describe ls command in docs
2024-01-27 13:22:00 +01:00
Michael Eischer
c90f24a06c
Merge pull request #4641 from MichaelEischer/reduce-restic-repository-usage
...
Misc cleanups
2024-01-27 13:18:20 +01:00
Michael Eischer
d4ed7c8858
walker: add tests for leaveDir
2024-01-27 13:17:33 +01:00
Michael Eischer
2c80cfa4a5
walker: fix missing leaveDir if directory is partially skipped
2024-01-27 13:17:33 +01:00
Michael Eischer
261737abc8
ls: only allow either --json or --ncdu
2024-01-27 13:17:33 +01:00
Michael Eischer
a2f2f8fb4c
fix linter warning
2024-01-27 13:17:33 +01:00
Michael Eischer
4bae54d040
ls: test ncdu output format
2024-01-27 13:17:33 +01:00
Michael Eischer
509b339d54
ls: correctly handle setuid/setgit/sticky bit in ncdu output
2024-01-27 13:17:33 +01:00
Michael Eischer
a2fe337610
ls: unify printer implementations
2024-01-27 13:17:33 +01:00
Michael Eischer
1b008c92d3
ls: rework ncdu output to use walker.LeaveDir
2024-01-27 13:17:33 +01:00
Michael Eischer
9ecbda059c
walker: add callback to inform about leaving a directory
2024-01-27 13:17:32 +01:00
Nils Decker
b2703a4089
add changelog for ls --ncdu
2024-01-27 13:17:15 +01:00
Nils Decker
a9310948cf
command ls: add option for ncdu output
...
NCDU (NCurses Disk Usage) is a tool to analyse disk usage of directories.
It has an option to save a directory tree and analyse it later.
This patch adds an output option to the ls command.
A snapshot can be seen with
`restic ls latest --ncdu | ncdu -f -`
- https://dev.yorhel.nl/ncdu
2024-01-27 13:06:52 +01:00
Michael Eischer
246559e654
check: cleanup s3 legacy detection
2024-01-27 13:02:04 +01:00
Michael Eischer
1dfd854769
lock: switch to repository.List
2024-01-27 13:02:04 +01:00
Michael Eischer
bfb56b78e1
replace some usages of restic.Repository with more specific interface
...
This should eventually make it easier to test the code.
2024-01-27 13:02:02 +01:00
Michael Eischer
3424088274
Merge pull request #4644 from MichaelEischer/refactor-repair-packs
...
Refactor and test `repair packs`
2024-01-27 13:00:51 +01:00
Michael Eischer
724ec179e3
Merge pull request #4648 from MichaelEischer/repository-removekey
...
repository: Introduce RemoveKey function
2024-01-27 13:00:42 +01:00
Michael Eischer
f0e1ad2285
fix linter warning
2024-01-27 12:51:45 +01:00
Michael Eischer
fd579421dd
repository: deduplicate test
2024-01-27 12:51:45 +01:00
Michael Eischer
42c9318b9c
repair pack: add tests
2024-01-27 12:51:45 +01:00
Michael Eischer
764b0bacd6
repair pack: add support for truncated files
2024-01-27 12:51:45 +01:00
Michael Eischer
7c351bc53c
repair pack: reenable auto index updates
...
The method is not available on the restic.Repository interface that is
used for testing. Drop the call as a small amount of additional index
writes is not a problem.
2024-01-27 12:51:45 +01:00
Michael Eischer
feeab84204
repair pack: extract the repair logic into the repository package
...
Currently, the cmd/restic package contains a significant amount of code
that modifies repository internals. This code should in the mid-term
move into the repository package.
2024-01-27 12:51:45 +01:00
Michael Eischer
d7a50fe739
properly show termstatus progress bar if visible less than one frame
...
If a progress bar using termstatus was only visible for less than one
frame, then its output could be lost.
2024-01-27 12:51:40 +01:00
Michael Eischer
6b65a495b1
backup/restore: fix termstatus initialization
...
The termstatus must only be canceled once the command has returned.
Otherwise output may be lost when the context gets canceled.
2024-01-27 12:51:08 +01:00
Michael Eischer
d26d2d41f8
backup/restore: extract termstatus initialization
2024-01-27 12:51:08 +01:00
Michael Eischer
cb50832d50
index: let MasterIndex.Save also delete obsolete indexes
2024-01-27 12:51:08 +01:00
Michael Eischer
bedff1ed6d
split deleteFiles into UI and logic parts
2024-01-27 12:51:08 +01:00
Michael Eischer
c13bf0b607
repository: Introduce RemoveKey function
...
This replaces directly removing keys via the backend.
2024-01-27 12:42:58 +01:00
Michael Eischer
25ac1549e7
Merge pull request #4661 from MichaelEischer/clarify-contributing
...
CONTRIBUTING.md: Clarify handling of small bugfixes
2024-01-26 23:04:58 +00:00
Michael Eischer
ae9683336d
CONTRIBUTING.md: Clarify handling of small bugfixes
...
Opening an issue for a small bugfix is usually not useful. It primarily
adds overhead.
2024-01-26 23:51:54 +01:00
Michael Eischer
446167ae80
Merge pull request #4643 from MichaelEischer/remove-redundant-poly1305-mask
...
Remove redundant poly1305 key masking
2024-01-23 19:46:40 +01:00
Michael Eischer
5b36c4eb5f
Merge pull request #4647 from MichaelEischer/reduce-globals
...
Remove all usages of the global command-specific options
2024-01-23 19:46:15 +01:00
Michael Eischer
1419baf67a
Merge pull request #4645 from MichaelEischer/improve-lock-checking
...
lock: checkForOtherLocks processes each lock at most once
2024-01-23 19:46:00 +01:00
Michael Eischer
66103aea3d
Remove all usages of the global command-specific options
...
Now, every command uses an options struct, which is passed to the run*
function by the command.RunE method.
2024-01-23 19:21:39 +01:00
Michael Eischer
79f2939eb9
Merge pull request #4654 from adrian5/docfix
...
docs: fix formatting
2024-01-23 18:08:59 +00:00
Michael Eischer
0e2ee06803
Merge pull request #4650 from MichaelEischer/improve-stdin-from-command-description
...
backup: Improve help text for `--stdin-from-command`
2024-01-23 19:22:15 +01:00
Michael Eischer
2927982256
Merge pull request #4649 from MichaelEischer/simplify-termstatus-shutdown
...
ui/termstatus: simplify cleaning up on termination
2024-01-23 19:16:21 +01:00
Michael Eischer
6cc2bec5dd
apply suggestion from review
2024-01-23 19:09:04 +01:00
Michael Eischer
18806944f6
doc: remove blockquotes from unordered lists
2024-01-23 19:03:54 +01:00
adrian5
609f84e095
docs: fix formatting
2024-01-22 21:12:12 +01:00
Michael Eischer
767c2539a0
backup: Improve help text for `--stdin-from-command`
2024-01-21 22:06:54 +01:00
Michael Eischer
6bdca13603
ui/termstatus: simplify cleaning up on termination
...
`writeStatus` also cleans no longer used status lines.
The old code actually cleaned one line too much. However, as that line
was never used it makes no difference.
2024-01-21 21:27:27 +01:00
Michael Eischer
f1f34eb3e5
lock: checkForOtherLocks processes each lock at most once
...
If a lock could not be loaded, then restic would check all lock files
again. These repeated checks are not useful as the status of a lock file
cannot change unless its ID changes too. Thus, skip already check lock
files on retries.
2024-01-20 22:40:12 +01:00
Michael Eischer
fee83e1c09
Remove redundant poly1305 key masking
...
The implementation in crypto/poly1305 already performs the exact same
masking.
2024-01-20 12:36:59 +01:00
Michael Eischer
6696195f38
Merge pull request #4584 from elkemper/fix-stop-archiving-metadata
...
S3: Don't archive metadata files on S3 Glacier
2024-01-20 10:30:37 +00:00
Michael Eischer
a763a5c67d
s3: minor cleanups for archive storage class handling
2024-01-20 11:25:28 +01:00
Vladislav Belous
8ca58b487c
S3: do not set storage class for metadata when using archive storage
2024-01-20 11:04:15 +01:00
Michael Eischer
62111f4379
Merge pull request #4625 from MichaelEischer/refactor-streampacks
...
Refactor repository.StreamPacks
2024-01-19 21:48:37 +01:00
Michael Eischer
2c310a526e
repository: Replace StreamPack function with LoadBlobsFromPack method
...
LoadBlobsFromPack is now part of the repository struct. This ensures
that users of that method don't have to deal will internals of the
repository implementation.
The filerestorer tests now also contain far fewer pack file
implementation details.
2024-01-19 21:40:43 +01:00
Michael Eischer
6b7b5c89e9
repository: prepare StreamPack refactor
2024-01-19 21:40:43 +01:00
Michael Eischer
22d0c3f8dc
check: Use PackBlobIterator instead of StreamPack
...
To only stream the content of a pack file once, check used StreamPack
with a custom pack load function. This combination was always brittle
and complicates using StreamPack everywhere else. Now that StreamPack
internally uses PackBlobIterator use that primitive instead, which is a
much better fit for what the check command requires.
2024-01-19 21:40:36 +01:00
Michael Eischer
fb422497af
repository: split StreamPack implementation
...
Move the actual decoding of the pack data into a separate iterator.
2024-01-19 21:39:55 +01:00
Michael Eischer
54c5c72e5a
Merge pull request #4616 from MichaelEischer/fix-rest-connection-close
...
rest: fix and cleanup closing of http response body
2024-01-19 21:31:35 +01:00
Michael Eischer
5f49eec655
Merge pull request #4615 from MichaelEischer/fix-find-empty-dirs
...
walker: Remove ignoreTrees functionality
2024-01-19 21:25:40 +01:00
Michael Eischer
ec13105093
Merge pull request #4623 from MichaelEischer/docs-verify-release-binaries
...
Add documentation for the verify-release-binaries.sh script
2024-01-19 21:17:38 +01:00
Michael Eischer
bd883caae1
CI: enable bodyclose linter
2024-01-19 21:17:18 +01:00
Michael Eischer
b1a8fd1d03
rest: fix and cleanup closing of http response body
...
If client.Do returns an error, then there's no body that has to be
closed. For requests for which we are not interested in the response
body, immediately drain and close the body to make sure it isn't
forgotten later on.
This change in particular adds the missing `Close()` call for the
`List()` command.
2024-01-19 21:17:17 +01:00
Michael Eischer
fdcbb53017
walker: test skipping for root node
2024-01-19 21:16:06 +01:00
Michael Eischer
0b39940fdb
walker: Remove ignoreTrees functionality
...
It was only used in two places:
- stats: apparently as a minor performance optimization, which is
unlikely to be important
- find: filtered directories would be ignored. However, this
optimization missed that it is possible that two directories have the
exact same content. Such directories would be incorrectly ignored too.
Example:
```
mkdir test test/a test/b
restic backup test
restic find latest test/b
-> incorrectly does not return anything
```
Thus, remove the functionality as it's apparently too complex to use
correctly.
2024-01-19 21:16:06 +01:00
Michael Eischer
147b0e54cb
Merge pull request #4639 from northben/patch-1
...
Update Backblaze documentation
2024-01-19 20:08:48 +00:00
Ben Northway
5413877d33
Update Backblaze documentation
...
clarify documentation regarding B2 bucket lifecycle settings. The default lifecycle setting is probably fine for most users now; a custom policy is not necessary.
2024-01-18 16:41:01 -06:00
Michael Eischer
03e06d0797
Merge branch 'patch-release'
2024-01-14 21:38:17 +01:00
Alexander Neumann
0ec9383ba2
Set development version for 0.16.3
2024-01-14 20:21:45 +01:00
Alexander Neumann
abca112404
Add version for 0.16.3
2024-01-14 20:21:45 +01:00
Alexander Neumann
b70b94507a
Generate CHANGELOG.md for 0.16.3
2024-01-14 20:21:19 +01:00
Alexander Neumann
d987582594
Prepare changelog for 0.16.3
2024-01-14 20:21:19 +01:00
Leo R. Lundgren
ef2e473b99
doc: Polish changelogs
2024-01-10 00:19:07 +01:00
Michael Eischer
7b2de84763
Merge pull request #4618 from MichaelEischer/workaround-rclone-list-errors
...
rclone: Workaround for incorrect "not found" errors while listing files
2024-01-09 18:28:31 +01:00
Michael Eischer
e4bbde7036
rclone: Workaround for incorrect "not found" errors while listing files
...
rclone returns a "not found" error if an internal error occurs while
listing a folder. Ignoring this error lets restic erroneously think that
there are no files, which can cause `prune` to wipe the whole
repository.
2024-01-09 18:28:17 +01:00
Michael Eischer
ec0fb46f6c
add changelog for reliable restores
2024-01-09 18:27:48 +01:00
Michael Eischer
103beb96bc
restore: separately restore blobs that are frequently referenced
...
Writing these blobs to their files can take a long time and consequently
cause the backend connection to time out. Avoid that by retrieving these
blobs separately.
2024-01-09 18:27:48 +01:00
Michael Eischer
f0f89d7f27
restore: split error reporting from downloadPack
2024-01-09 18:27:48 +01:00
Michael Eischer
cf352ccafb
restore: cleanup downloadPack
2024-01-09 18:27:48 +01:00
Michael Eischer
b856e9489a
restore: split downloadPack into smaller methods
2024-01-09 18:27:48 +01:00
Michael Eischer
c31e9418ba
Merge pull request #4626 from MichaelEischer/reliable-large-restores
...
Improve reliability of large restores
2024-01-09 18:23:09 +01:00
Michael Eischer
2e8de9edfd
rclone: Workaround for incorrect "not found" errors while listing files
...
rclone returns a "not found" error if an internal error occurs while
listing a folder. Ignoring this error lets restic erroneously think that
there are no files, which can cause `prune` to wipe the whole
repository.
2024-01-09 18:20:16 +01:00
Michael Eischer
ce7db90e08
sync CI and go dependencies with master branch
2024-01-08 21:33:05 +01:00
Michael Eischer
620518aec6
add changelog for better restore error reporting
2024-01-08 21:33:05 +01:00
Michael Eischer
f2fafbffaa
restore: only report errors for blobs that actually failed to load
...
Previously, errors would be reported for all blobs of a packfile that
failed to stream. Now, only the not yet processed blobs are reported.
2024-01-08 21:33:05 +01:00
Michael Eischer
7a3a884874
repository: test that StreamPack only delivers blobs once
2024-01-08 21:33:05 +01:00
Michael Eischer
772a907533
repository: StreamPack delivers blobs at most once
...
If an error occurred while streaming a pack file, this could result in
passing some of the blobs multiple times to the callback function. This
significantly complicates using StreamPack correctly and is unnecessary.
Retries do not change the content of a blob and thus only deliver the
same result over and over again.
2024-01-08 21:33:05 +01:00
Michael Eischer
a9446c1184
add changelog for irregular files on windows
2024-01-08 21:33:05 +01:00
Michael Eischer
1bab29c336
archiver: Add filepath to error message if it is not included yet
2024-01-08 21:33:05 +01:00
Michael Eischer
e886c3f6b2
archiver: improve error message for irregular files
...
Since Go 1.21, most reparse points are considered as irregular files.
Depending on the underlying driver these can exhibit nearly arbitrary
behavior. When encountering such a file, restic returned an
indecipherable error message: `error: invalid node type ""`.
Add the filepath to the error message and state that the file type is
not supported.
2024-01-08 21:33:05 +01:00
Michael Eischer
c95de54726
restic: cleanup node type determination
...
os.ModeCharDevice is already included in os.ModeType
2024-01-08 21:33:05 +01:00
Michael Eischer
d4b8abd3e2
fix deduplicated files on windows
2024-01-08 21:33:05 +01:00
Joram Berger
948ab3ccaf
Add a note that the oldest snapshot may be kept additionally
...
Documentation enhancement.
2024-01-08 21:33:05 +01:00
Markus Zoppelt
bb0c923298
docs: add pkgx install option
...
PR in pkgx pantry: https://github.com/pkgxdev/pantry/pull/4098
restic pkg:
https://pkgx.dev/pkgs/restic.net/restic/
2024-01-08 21:33:05 +01:00
Michael Eischer
ff0c975443
regenerate changelog
2024-01-08 21:33:05 +01:00
Michael Eischer
7e61e117d6
cleanup changelog whitespace
2024-01-08 21:33:05 +01:00
mmattel
220a28582e
Add a table of contents (TOC) to the changelog template
2024-01-08 21:33:05 +01:00
Giuseppe D'Andrea
f44fd73230
docs: fix typo in working with repos
...
When using the `copy` command, `--from-password-file` and `--from-password-command` flags are used to specify the password of the source repository, not of the destination repository.
2024-01-08 21:33:05 +01:00
Joda Stößer
76bd975e03
docs(scripting): correct stats output comment to be about the correct command
...
not about the snapshots command
2024-01-08 21:33:05 +01:00
Quang-Linh LE
64b7aed362
docs: Mention progress for restore command. This is available after https://github.com/restic/restic/pull/3991
2024-01-08 21:33:05 +01:00
Michael Kuhn
3fa6b2de4a
Fix repository not being printed when using repository file
...
When using `RESTIC_REPOSITORY_FILE` in combination with `restic init`,
the repository is missing in the output:
```
$ restic init
created restic repository 3c872be20f at
[...]
```
This is due to the code using `gopts.Repo`, which is empty in this case.
2024-01-08 21:33:05 +01:00
Michael Eischer
5cd000f4b0
CI: update golangci-lint
...
Necessary to properly support Go 1.21.
2024-01-08 21:33:05 +01:00
Michael Eischer
4ea3796455
add changelog for reliable restores
2024-01-08 21:03:10 +01:00
Michael Eischer
e78be75d1e
restore: separately restore blobs that are frequently referenced
...
Writing these blobs to their files can take a long time and consequently
cause the backend connection to time out. Avoid that by retrieving these
blobs separately.
2024-01-08 21:00:13 +01:00
Michael Eischer
2267910418
restore: split error reporting from downloadPack
2024-01-08 20:57:00 +01:00
Michael Eischer
00d18b7a88
restore: cleanup downloadPack
2024-01-08 20:53:08 +01:00
Michael Eischer
9328f34d43
restore: split downloadPack into smaller methods
2024-01-08 20:52:36 +01:00
Michael Eischer
77434c6e2b
Merge pull request #4474 from ekristen/aws-assume-role
...
Allow AWS Assume Role
2024-01-08 19:07:17 +00:00
Michael Eischer
4248c6c3ca
s3: update documentation
2024-01-07 19:30:11 +01:00
Michael Eischer
e4a7eb09ef
Merge pull request #4624 from MichaelEischer/better-restorer-error-reporting
...
Improver restorer error reporting
2024-01-07 11:20:29 +01:00
Michael Eischer
f8b4e932ef
Merge pull request #4620 from MichaelEischer/improve-irregular-file-handling
...
Improve irregular file handling
2024-01-07 11:12:07 +01:00
Michael Eischer
100872308f
add changelog for better restore error reporting
2024-01-07 11:06:42 +01:00
Michael Eischer
dac3508170
restore: only report errors for blobs that actually failed to load
...
Previously, errors would be reported for all blobs of a packfile that
failed to stream. Now, only the not yet processed blobs are reported.
2024-01-07 10:54:56 +01:00
Michael Eischer
77b1c52673
repository: test that StreamPack only delivers blobs once
2024-01-07 10:54:53 +01:00
Michael Eischer
fe5c337ca2
repository: StreamPack delivers blobs at most once
...
If an error occurred while streaming a pack file, this could result in
passing some of the blobs multiple times to the callback function. This
significantly complicates using StreamPack correctly and is unnecessary.
Retries do not change the content of a blob and thus only deliver the
same result over and over again.
2024-01-07 10:54:49 +01:00
Michael Eischer
3e29f8dddf
add changelog for irregular files on windows
2024-01-07 10:52:12 +01:00
Michael Eischer
76f507c775
Merge pull request #4621 from MichaelEischer/fix-windows-dedup-files
...
Fix backup of deduplicated files on windows
2024-01-07 10:44:50 +01:00
Michael Eischer
6ef23b401b
fix deduplicated files on windows
2024-01-07 10:23:31 +01:00
Michael Eischer
62f99a3b2f
Add documentation for the verify-release-binaries.sh script
2024-01-07 10:16:30 +01:00
Michael Eischer
0360e540af
Merge pull request #4622 from MichaelEischer/fix-outdated-windows-import
...
termstatus: update import path of golang.org/x/term
2024-01-06 23:24:16 +01:00
Michael Eischer
e6dfefba13
termstatus: update import path of golang.org/x/term
2024-01-06 21:59:26 +01:00
Michael Eischer
02bc73f5eb
s3: minor code cleanups
2024-01-06 21:44:53 +01:00
Michael Eischer
20cf4777cb
s3: check for EnvAWS credentials before Static credentials
...
EnvAWS considers more environment variables, including AWS_SESSION_TOKEN
and thus should be checked first.
2024-01-06 21:43:47 +01:00
Erik Kristensen
5ffb536aae
feat: support AWS assume role
2024-01-06 21:19:58 +01:00
Michael Eischer
1604922360
Merge pull request #4527 from adamantike/cmd/copy/prefix-hostname-to-snapshot-paths
...
cmd: Add hostname to snapshot display output
2024-01-06 19:32:14 +00:00
Michael Eischer
c7844530d8
update docs
2024-01-06 20:25:24 +01:00
Michael Eischer
33b7c84a7a
deduplicate string formatting of snapshot metadata
...
This removes the spurious ")" bracket at the end and normalizes the
metadata format used by the `ls` command.
2024-01-06 20:20:51 +01:00
Michael Manganiello
045aa64558
cmd/copy: Prefix hostname to snapshot display output
...
This change better resembles the output generated by `Snapshot.String()`,
which includes both username and hostname.
Closes #4506
Before:
```
$ restic copy --from-repo /srv/restic-repo
repository 3666882b opened (version 2, compression level auto)
repository 0085c387 opened (version 2, compression level auto)
created new cache in /home/mike/.cache/restic
[0:00] 100.00% 1 / 1 index files loaded
[0:00] 0 index files loaded
snapshot 32b39a20 of [/home/mike/data] at 2023-10-21 16:01:13.979948154 -0300 -03)
copy started, this may take a while...
[0:00] 100.00% 1 / 1 packs copied
snapshot 10331fdd saved
```
After:
```
$ restic copy --from-repo /srv/restic-repo
repository 3666882b opened (version 2, compression level auto)
repository 0085c387 opened (version 2, compression level auto)
[0:00] 100.00% 1 / 1 index files loaded
[0:00] 0 index files loaded
snapshot 32b39a20 of [/home/mike/data] at 2023-10-21 16:01:13.979948154 -0300 -03 by mike@desktop)
copy started, this may take a while...
[0:00] 100.00% 1 / 1 packs copied
snapshot a67bd1ee saved
```
2024-01-06 20:20:46 +01:00
Michael Eischer
b2b7669ca0
Merge pull request #4526 from dnnr/detect-bitrot-in-diff
...
Add bitrot detection to "diff" command
2024-01-06 19:18:34 +00:00
Michael Eischer
4f6b1bb6f6
diff: document limitations regarding metadata
2024-01-06 20:12:47 +01:00
Michael Eischer
3549635243
diff: copy nodes before modifying them for bitrot detection
2024-01-06 20:12:47 +01:00
Daniel Danner
a7dc18e697
Add bitrot detection to "diff" command
...
This introduces a new modifier to the output of the diff command. It
appears whenever two files being compared only differ in their content
but not in their metadata. As far as we know, under normal
circumstances, this should only ever happen if some kind of bitrot has
happened in the source file. The prerequisite for this detection to work
is that the right-side snapshot of the comparison has been created with
"backup --force".
2024-01-06 20:12:47 +01:00
Michael Eischer
51419c51d3
archiver: Add filepath to error message if it is not included yet
2024-01-06 19:08:24 +01:00
Michael Eischer
6b79834cc8
archiver: improve error message for irregular files
...
Since Go 1.21, most reparse points are considered as irregular files.
Depending on the underlying driver these can exhibit nearly arbitrary
behavior. When encountering such a file, restic returned an
indecipherable error message: `error: invalid node type ""`.
Add the filepath to the error message and state that the file type is
not supported.
2024-01-06 19:03:11 +01:00
Michael Eischer
0018bb7854
restic: cleanup node type determination
...
os.ModeCharDevice is already included in os.ModeType
2024-01-06 18:43:16 +01:00
Michael Eischer
634e2a46d9
Merge pull request #4608 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.66
...
build(deps): bump github.com/minio/minio-go/v7 from 7.0.63 to 7.0.66
2024-01-06 11:40:02 +00:00
Michael Eischer
dfcab92db2
Merge pull request #4609 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.9.1
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.8.0 to 1.9.1
2024-01-06 11:06:07 +00:00
dependabot[bot]
3666eef76c
build(deps): bump github.com/minio/minio-go/v7 from 7.0.63 to 7.0.66
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.63 to 7.0.66.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.63...v7.0.66 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-01-06 11:03:52 +00:00
Michael Eischer
3a61622dfe
Merge pull request #4610 from restic/dependabot/github_actions/actions/setup-go-5
...
build(deps): bump actions/setup-go from 4 to 5
2024-01-06 11:01:07 +00:00
Michael Eischer
5c4fca76df
Merge pull request #4607 from restic/dependabot/go_modules/golang.org/x/crypto-0.17.0
...
build(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0
2024-01-06 10:56:40 +00:00
Michael Eischer
98da0bdd12
Merge pull request #4606 from restic/dependabot/go_modules/github.com/klauspost/compress-1.17.4
...
build(deps): bump github.com/klauspost/compress from 1.17.2 to 1.17.4
2024-01-06 10:55:00 +00:00
dependabot[bot]
2c60dd97ae
build(deps): bump actions/setup-go from 4 to 5
...
Bumps [actions/setup-go](https://github.com/actions/setup-go ) from 4 to 5.
- [Release notes](https://github.com/actions/setup-go/releases )
- [Commits](https://github.com/actions/setup-go/compare/v4...v5 )
---
updated-dependencies:
- dependency-name: actions/setup-go
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 01:33:20 +00:00
dependabot[bot]
40905403f4
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.8.0...sdk/azcore/v1.9.1 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 01:18:36 +00:00
dependabot[bot]
7e7cbe8e19
build(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0
...
Bumps [golang.org/x/crypto](https://github.com/golang/crypto ) from 0.16.0 to 0.17.0.
- [Commits](https://github.com/golang/crypto/compare/v0.16.0...v0.17.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/crypto
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 01:18:08 +00:00
dependabot[bot]
44646c20be
build(deps): bump github.com/klauspost/compress from 1.17.2 to 1.17.4
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.17.2 to 1.17.4.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.17.2...v1.17.4 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2024-01-01 01:17:57 +00:00
Michael Eischer
8f9a35779e
Merge pull request #4600 from MichaelEischer/doc-forget-note-1
...
Add a note that the oldest snapshot may be kept additionally
2023-12-29 17:56:54 +01:00
Michael Eischer
23e1b4bbb1
Merge pull request #4573 from gab50000/rewrite_time
...
Rewrite metadata
2023-12-24 14:42:00 +00:00
Michael Eischer
01b33734ab
rewrite: update command output in docs
2023-12-24 15:36:22 +01:00
Michael Eischer
649a6409ee
rewrite: cleanup tests
2023-12-24 15:36:22 +01:00
Michael Eischer
c31f5f986c
rewrite: Minor cleanups
2023-12-24 15:36:22 +01:00
Michael Eischer
2730d05fce
rewrite: Don't walk snapshot content if only metadata is modified
2023-12-24 15:36:22 +01:00
Michael Eischer
893d0d6325
rewrite: cleanup new metadata options and fix no parameters check
2023-12-24 15:36:22 +01:00
Gabriel Kabbe
7de97d7480
rewrite: Add documentation
2023-12-24 15:36:22 +01:00
Gabriel Kabbe
004520a238
rewrite: Add changelog
2023-12-24 15:36:22 +01:00
Gabriel Kabbe
a02d8d75c2
rewrite: Implement rewriting metadata
2023-12-24 15:36:22 +01:00
Gabriel Kabbe
7bf38b6c50
rewrite: Add test TestRewriteMetadata
2023-12-24 15:36:22 +01:00
Gabriel Kabbe
da1704b2d5
rewrite: Add tests
...
Pass nil instead of metadata to existing tests
2023-12-24 15:36:19 +01:00
Gabriel Kabbe
3026baea07
rewrite: Add structs for tracking metadata changes
...
Adds
* snapshotMetadataArgs, which holds the new metadata as strings parsed from
the command line
* snapshotMetadata, which holds the new metadata converted to the
correct types
2023-12-24 14:43:07 +01:00
Michael Eischer
1196c72819
Merge pull request #4570 from MarkusZoppelt/docs/pkgx
...
docs: add pkgx install option
2023-12-24 11:04:59 +00:00
Michael Eischer
433dd92959
Merge pull request #4590 from renard/optimize-mount-failure
...
mount: detect mountpoint does not exist before opening the repository
2023-12-24 10:59:56 +00:00
Markus Zoppelt
c14740c50f
docs: add pkgx install option
...
PR in pkgx pantry: https://github.com/pkgxdev/pantry/pull/4098
restic pkg:
https://pkgx.dev/pkgs/restic.net/restic/
2023-12-24 11:59:12 +01:00
Michael Eischer
5537460664
Merge pull request #4598 from MichaelEischer/add_table_of_contents
...
Add table of contents
2023-12-24 11:56:05 +01:00
Sébastien Gross
f7587be28f
mount: detect mountpoint does not exist before opening the repository
...
Bug #1681 suggests that restic should not be nice to user and should
refrain from creating a mountpoint if it does not exist. Nevertheless,
it currently opens the repository before checking for the mountpoint's
existence. In the case of large or remote repositories, this process
can be time-consuming, delaying the inevitable outcome.
/restic mount --repo=REMOTE --verbose /tmp/backup
repository 33f14e42 opened (version 2, compression level max)
[0:38] 100.00% 162 / 162 index files loaded
Mountpoint /tmp/backup doesn't exist
stat /tmp/backup: no such file or directory
real 0m39.534s
user 1m53.961s
sys 0m3.044s
In this scenario, 40 seconds could have been saved if the nonexistence
of the path had been verified beforehand.
This patch relocates the mountpoint check to the beginning of the
runMount function, preceding the opening of the repository.
/restic mount --repo=REMOTE --verbose /tmp/backup
Mountpoint /tmp/backup doesn't exist
stat /tmp/backup: no such file or directory
real 0m0.136s
user 0m0.018s
sys 0m0.027s
Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
2023-12-24 11:54:18 +01:00
Michael Eischer
91fb703756
regenerate changelog
2023-12-24 11:47:31 +01:00
Michael Eischer
d7ff862b8d
cleanup changelog whitespace
2023-12-24 11:47:31 +01:00
mmattel
db1d920c80
Add a table of contents (TOC) to the changelog template
2023-12-24 11:47:31 +01:00
Michael Eischer
c6299f8dbd
Merge pull request #4582 from giuseppedandrea/docs/fix-typo
...
docs: fix typo in working with repos
2023-12-24 00:09:59 +00:00
Giuseppe D'Andrea
a128976014
docs: fix typo in working with repos
...
When using the `copy` command, `--from-password-file` and `--from-password-command` flags are used to specify the password of the source repository, not of the destination repository.
2023-12-24 01:04:36 +01:00
Michael Eischer
e2f6109a52
Merge pull request #4580 from restic/dependabot/go_modules/golang.org/x/oauth2-0.15.0
...
build(deps): bump golang.org/x/oauth2 from 0.13.0 to 0.15.0
2023-12-24 00:02:30 +00:00
Michael Eischer
30e6ed038c
Merge pull request #4586 from Gelma/typos
...
Fix typos
2023-12-23 13:21:50 +00:00
Michael Eischer
e96d1ee33e
Merge pull request #4593 from SimJoSt/patch-1
...
docs(scripting): correct stats output comment to be about the correct command
2023-12-23 13:09:20 +00:00
dependabot[bot]
0054db394f
build(deps): bump golang.org/x/oauth2 from 0.13.0 to 0.15.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.13.0 to 0.15.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.13.0...v0.15.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-12-23 13:08:29 +00:00
Michael Eischer
53ebe91a50
Move changelog typo fixes to changelog files
2023-12-23 14:05:47 +01:00
Michael Eischer
9ceaea34dd
Merge pull request #4572 from linktohack/patch-1
...
docs: Mention progress for restore command. This is available after https://github.com/restic/restic/pull/3991
2023-12-23 13:00:32 +00:00
Michael Eischer
356b7aac16
Merge pull request #4571 from michaelkuhn/init-repo-file
...
Fix repository not being printed when using repository file
2023-12-23 12:59:06 +00:00
Joda Stößer
eef7c65655
docs(scripting): correct stats output comment to be about the correct command
...
not about the snapshots command
2023-12-23 13:56:40 +01:00
Michael Eischer
97b8629336
Merge pull request #4579 from restic/dependabot/go_modules/golang.org/x/sync-0.5.0
...
build(deps): bump golang.org/x/sync from 0.4.0 to 0.5.0
2023-12-23 12:51:52 +00:00
Michael Eischer
d2ecd6bef2
Merge pull request #4577 from restic/dependabot/go_modules/golang.org/x/time-0.5.0
...
build(deps): bump golang.org/x/time from 0.3.0 to 0.5.0
2023-12-23 12:50:59 +00:00
Michael Eischer
634750a732
Merge pull request #4576 from restic/dependabot/github_actions/docker/login-action-3d58c274f17dffee475a5520cbe67f0a882c4dbb
...
build(deps): bump docker/login-action from 1f401f745bf57e30b3a2800ad308a87d2ebdf14b to 3d58c274f17dffee475a5520cbe67f0a882c4dbb
2023-12-23 12:50:52 +00:00
Quang-Linh LE
c554825e2d
docs: Mention progress for restore command. This is available after https://github.com/restic/restic/pull/3991
2023-12-23 13:50:20 +01:00
Michael Kuhn
fd2fb233aa
Fix repository not being printed when using repository file
...
When using `RESTIC_REPOSITORY_FILE` in combination with `restic init`,
the repository is missing in the output:
```
$ restic init
created restic repository 3c872be20f at
[...]
```
This is due to the code using `gopts.Repo`, which is empty in this case.
2023-12-23 13:49:22 +01:00
Michael Eischer
da4e3edbbc
Merge pull request #4596 from MichaelEischer/update-golangcilint
...
CI: update golangci-lint
2023-12-23 13:48:33 +01:00
Michael Eischer
dbbd31bc3a
CI: update golangci-lint
...
Necessary to properly support Go 1.21.
2023-12-23 13:41:30 +01:00
Joram Berger
12af20e606
Add a note that the oldest snapshot may be kept additionally
...
Documentation enhancement.
2023-12-18 18:24:57 +01:00
Andrea Gelmini
241916d55b
Fix typos
2023-12-06 13:11:55 +01:00
dependabot[bot]
427b90cf82
build(deps): bump golang.org/x/sync from 0.4.0 to 0.5.0
...
Bumps [golang.org/x/sync](https://github.com/golang/sync ) from 0.4.0 to 0.5.0.
- [Commits](https://github.com/golang/sync/compare/v0.4.0...v0.5.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sync
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-12-01 01:15:09 +00:00
dependabot[bot]
eec6e014f4
build(deps): bump golang.org/x/time from 0.3.0 to 0.5.0
...
Bumps [golang.org/x/time](https://github.com/golang/time ) from 0.3.0 to 0.5.0.
- [Commits](https://github.com/golang/time/compare/v0.3.0...v0.5.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/time
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-12-01 01:14:53 +00:00
dependabot[bot]
fa46a47e22
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from 1f401f745bf57e30b3a2800ad308a87d2ebdf14b to 3d58c274f17dffee475a5520cbe67f0a882c4dbb.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](1f401f745b...3d58c274f1
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-12-01 01:02:16 +00:00
Michael Eischer
b72de5a883
Merge branch 'patch-release'
2023-11-12 11:52:21 +01:00
Michael Eischer
6086ae4ca7
Merge pull request #4563 from smlx/mac-signature
...
fix: drop reference to signature and define MAC
2023-11-08 21:59:13 +00:00
Scott Leggett
aeaf527be1
fix: drop reference to signature and define MAC
...
Poly1305-AES is not a signature, so don't mention that.
In addition, the term MAC was used without being defined, so add a
definition.
Signed-off-by: Scott Leggett <scott@sl.id.au>
2023-11-06 20:12:42 +08:00
Michael Eischer
19068aa82f
Merge pull request #4559 from DRON-666/html-zip
...
Restore generation of `HTMLZip` files on `readthedocs.com`.
2023-11-02 20:53:12 +00:00
Michael Eischer
81ca9d28f2
Merge pull request #4553 from CommanderRoot/add-version-json
...
Add --json option to version command
2023-11-01 21:58:38 +00:00
Tobias Speicher
ce53ea32c6
Split `go_target` into `go_os` and `go_arch`
2023-11-01 22:43:38 +01:00
Tobias Speicher
10cbc169c1
Use different function to be more consistent with other code
2023-11-01 22:18:37 +01:00
DRON-666
03f8f494e9
doc: add HTMLZip format to .readthedocs.yaml
2023-11-02 00:16:47 +03:00
Tobias Speicher
ab23d033b6
Add version command output to JSON format documentation
2023-11-01 22:13:57 +01:00
Tobias Speicher
6f1efcb28b
Update wording on changelog entry
2023-11-01 22:12:19 +01:00
Michael Eischer
9c399e55e3
Merge pull request #4554 from restic/dependabot/github_actions/docker/login-action-1f401f745bf57e30b3a2800ad308a87d2ebdf14b
...
build(deps): bump docker/login-action from b4bedf8053341df3b5a9f9e0f2cf4e79e27360c6 to 1f401f745bf57e30b3a2800ad308a87d2ebdf14b
2023-11-01 20:42:11 +00:00
Michael Eischer
e550bc0713
Merge pull request #4555 from restic/dependabot/go_modules/google.golang.org/api-0.149.0
...
build(deps): bump google.golang.org/api from 0.148.0 to 0.149.0
2023-11-01 20:40:45 +00:00
Michael Eischer
28aa9826af
Merge pull request #4557 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.34.0
...
build(deps): bump cloud.google.com/go/storage from 1.33.0 to 1.34.0
2023-11-01 20:40:40 +00:00
dependabot[bot]
6dde019ac8
build(deps): bump cloud.google.com/go/storage from 1.33.0 to 1.34.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.33.0 to 1.34.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.33.0...spanner/v1.34.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-11-01 01:45:02 +00:00
dependabot[bot]
c2f9e21d3c
build(deps): bump google.golang.org/api from 0.148.0 to 0.149.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.148.0 to 0.149.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.148.0...v0.149.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-11-01 01:44:28 +00:00
dependabot[bot]
67e6b9104a
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from b4bedf8053341df3b5a9f9e0f2cf4e79e27360c6 to 1f401f745bf57e30b3a2800ad308a87d2ebdf14b.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](b4bedf8053...1f401f745b
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-11-01 01:25:22 +00:00
Tobias Speicher
6ca07ee004
add changelog/unreleases for issue-4547
2023-10-31 19:39:52 +01:00
Tobias Speicher
d45cc52468
command version: add json output option
2023-10-31 19:31:07 +01:00
Alexander Neumann
59fe24cb2b
Set development version for 0.16.2
2023-10-29 19:50:12 +01:00
Alexander Neumann
1a5efcf680
Add version for 0.16.2
2023-10-29 19:50:07 +01:00
Alexander Neumann
d33fe6dd3c
Generate CHANGELOG.md for 0.16.2
2023-10-29 19:50:04 +01:00
Alexander Neumann
c8dd95f104
Prepare changelog for 0.16.2
2023-10-29 19:50:04 +01:00
Leo R. Lundgren
7d980b469d
doc: Polish changelogs
2023-10-29 00:44:53 +02:00
Michael Eischer
d863234e3e
add changelog for missing documentation
2023-10-28 20:25:24 +02:00
Michael Eischer
42ab3ea2b9
Merge pull request #4410 from Enrico204/restic-stdin-command
...
add --stdin-from-command flag to backup command
2023-10-27 22:09:06 +00:00
Michael Eischer
be28a02626
doc: tweak description for --stdin-from-command
2023-10-27 23:58:52 +02:00
Michael Eischer
5d152c7720
extend changelog for --stdin-from-command
2023-10-27 23:58:52 +02:00
Michael Eischer
ee305e6041
backup: rework error reporting for subcommand
2023-10-27 23:58:52 +02:00
Michael Eischer
8bceb8e359
fs: add tests for CommandReader
2023-10-27 23:58:52 +02:00
Michael Eischer
317144c1d6
fs: merge command startup into CommandReader
2023-10-27 23:58:51 +02:00
Michael Eischer
7d879705ad
fs: cleanup CommandReader implementation
2023-10-27 23:58:51 +02:00
Enrico204
37a312e505
restic-from-command: use standard behavior when no output and exit code 0 from command
...
The behavior of the new option should reflect the behavior of normal backups: when the command exit code is zero and there is no output in the stdout, emit a warning but create the snapshot. This commit fixes the integration tests and the ReadCloserCommand struct.
2023-10-27 23:58:51 +02:00
Enrico204
c0ca54dc8a
restic-from-command: add tests
2023-10-27 23:58:51 +02:00
Enrico204
81f8d473df
restic-from-command: abort snapshot on non-zero exit codes
2023-10-27 23:58:51 +02:00
Enrico204
6990b0122e
Add issue-4251 (stdin-from-command) in the changelog
2023-10-27 23:58:51 +02:00
Enrico204
072b227544
stdin-from-command: add documentation in backup sub-command
2023-10-27 23:58:51 +02:00
Enrico204
4e5caab114
stdin-from-command: implemented suggestions in #4254
...
The code has been refactored so that the archiver is back to the original code, and the stderr is handled using a go routine to avoid deadlock.
2023-10-27 23:58:51 +02:00
Sebastian Hoß
c133065a9f
Check command result before snapshotting
...
Return with an error containing the stderr of the given command in case it fails. No new snapshot will be created and future prune operations on the repository will remove the unreferenced data.
Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
25350a9c55
Extend SnapshotOptions w/ command data
...
In order to determine whether to save a snapshot, we need to capture the exit code returned by a command. In order to provide a nice error message, we supply stderr as well.
Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
a2b76ff34f
Start command from --stdin-from-command
...
It acts similar to --stdin but reads its data from the stdout of the given command instead of os.Stdin.
Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
333fe1c3cf
Align Stdin and StdinCommand in conditionals
...
In order to run with --stdin-from-command we need to short-circuit some functions similar to how it is handled for the --stdin flag. The only difference here is that --stdin-from-command actually expects that len(args) should be greater 0 whereas --stdin does not expect any args at all.
Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Sebastian Hoß
a8657bde68
Add --stdin-from-command option
...
This new flag is added to the backup subcommand in order to allow restic to control the execution of a command and determine whether to save a snapshot if the given command succeeds.
Signed-off-by: Sebastian Hoß <seb@xn--ho-hia.de>
2023-10-27 23:58:51 +02:00
Michael Eischer
104107886a
Merge pull request #4503 from MichaelEischer/fix-stats-with-hardlinks
...
stats: Fix hardlink tracking across multiple filesystems
2023-10-27 23:53:03 +02:00
Michael Eischer
731b3a4357
stats: fix hardlink tracking in a snapshot
...
inodes are only unique within a device. Use the HardlinkIndex from the
restorer instead of the custom (broken) hashmap to correctly account for
both inode and deviceID.
2023-10-27 23:40:42 +02:00
Michael Eischer
a8fdcf79b7
restorer: Make hardlink index generic
...
This will allow reusing it for the stats command without regressing the
memory usage due to storing an unnecessary file path.
2023-10-27 23:40:42 +02:00
Michael Eischer
45962c2847
Merge pull request #4499 from MichaelEischer/modular-backend-code
...
Split backend code from restic package
2023-10-27 20:19:20 +02:00
Michael Eischer
4be45de1c2
Restore support for ARMv5 platforms
2023-10-27 19:52:14 +02:00
Leo R. Lundgren
8c1125fe13
doc: Correct two typos
2023-10-27 19:48:00 +02:00
Martin Michlmayr
0b6ccea461
Fix typos
2023-10-27 19:48:00 +02:00
Martin Michlmayr
de6135351e
Format option correctly
2023-10-27 19:48:00 +02:00
Michael Eischer
d47581b25e
verify-release-binaries.sh: don't show warning if binaries are correct
2023-10-27 19:48:00 +02:00
Michael Eischer
69dec02a14
Remove readthedocs special case from docs configuration
...
Apparently it's now required to bring your own theme.
2023-10-27 19:47:19 +02:00
Michael Eischer
826d880614
Fix doc path typo in readthedocs configuration
2023-10-27 19:47:15 +02:00
Michael Eischer
dbf7ef72b9
Add read the docs config file version 2
...
The config file is by now necessary to build documentation:
https://blog.readthedocs.com/migrate-configuration-v2/
2023-10-27 19:47:11 +02:00
Michael Eischer
50ef01131a
Merge pull request #4542 from MichaelEischer/support-armv6
...
Only support ARMv6 on ARM platforms
2023-10-27 19:42:49 +02:00
rawtaz
6be3a8fe51
Merge pull request #4539 from tbm/docs
...
Fix typos in docs
2023-10-27 17:30:08 +00:00
Michael Eischer
5166bde386
Only support ARMv6 on ARM platforms
...
Go 1.21 has switched the default from GOARM=5 to GOARM=7. So far there
have been complaints from Raspberry Pi 1 users, as the first raspberry
pi version only supports ARMv6. Exclude older ARM versions as these are
likely not relevant (rest-server also only supports ARMv6/7) and enforce
the use of software floating point emulation.
2023-10-27 19:12:12 +02:00
Leo R. Lundgren
aafb806a8c
doc: Correct two typos
2023-10-27 18:56:32 +02:00
Martin Michlmayr
41e6a02bcc
Fix typos
2023-10-27 18:56:32 +02:00
Martin Michlmayr
b51fe2fb69
Format option correctly
2023-10-27 18:56:32 +02:00
Michael Eischer
56537fb48e
Merge pull request #4545 from restic/fix-rtd3
...
Try to fix documentation build
2023-10-26 22:00:20 +02:00
Michael Eischer
feea567868
Remove readthedocs special case from docs configuration
...
Apparently it's now required to bring your own theme.
2023-10-26 21:56:36 +02:00
Michael Eischer
2968b52f84
Merge pull request #4543 from MichaelEischer/fix-rtd2
...
Fix doc path typo in readthedocs configuration
2023-10-26 20:40:45 +02:00
Michael Eischer
619e80d7cc
Fix doc path typo in readthedocs configuration
2023-10-26 20:39:43 +02:00
Michael Eischer
3804e50d64
Merge pull request #4541 from MichaelEischer/fix-binary-check-script
...
verify-release-binaries.sh: don't show warning if binaries are correct
2023-10-26 20:37:56 +02:00
Michael Eischer
c19e39968f
verify-release-binaries.sh: don't show warning if binaries are correct
2023-10-26 19:59:27 +02:00
Michael Eischer
550be5c1e9
Merge pull request #4538 from MichaelEischer/fix-rtd
...
Add read the docs config file version 2
2023-10-26 19:42:36 +02:00
Michael Eischer
249605843b
prune: get backend connection count via repository
2023-10-25 23:01:54 +02:00
Michael Eischer
c7b770eb1f
convert MemorizeList to be repository based
...
Ideally, code that uses a repository shouldn't directly interact with
the underlying backend. Thus, move MemorizeList one layer up.
2023-10-25 23:01:35 +02:00
Michael Eischer
1b8a67fe76
move Backend interface to backend package
2023-10-25 23:00:18 +02:00
Michael Eischer
ceb0774af1
backend: make LoadAll independent of restic package
2023-10-25 22:58:39 +02:00
Michael Eischer
b6d79bdf6f
restic: decouple restic.Handle
2023-10-25 22:54:07 +02:00
Michael Eischer
7881309d63
backend: move backend implementation helpers to util package
...
This removes code that is only used within a backend implementation from
the backend package. The latter now only contains code that also has
external users.
2023-10-25 22:54:07 +02:00
Michael Eischer
8e6fdf5edf
Merge pull request #4520 from awannabeengineer/load-retry-nonexistent
...
retry: Do not retry Load() if file does not exist
2023-10-25 20:42:05 +00:00
Michael Eischer
c2e3e8d6ea
Add read the docs config file version 2
...
The config file is by now necessary to build documentation:
https://blog.readthedocs.com/migrate-configuration-v2/
2023-10-25 22:00:42 +02:00
Alexander Neumann
27ec320eae
Set development version for 0.16.1
2023-10-24 20:02:58 +02:00
Alexander Neumann
baca3f6303
Add version for 0.16.1
2023-10-24 20:02:55 +02:00
Alexander Neumann
524c2721b4
Update manpages and auto-completion
2023-10-24 20:02:55 +02:00
Alexander Neumann
be1b978ac8
Generate CHANGELOG.md for 0.16.1
2023-10-24 20:02:29 +02:00
Alexander Neumann
d4d9c1b8f1
Prepare changelog for 0.16.1
2023-10-24 20:02:29 +02:00
Michael Eischer
ead8dd0173
Merge pull request #4524 from MichaelEischer/fix-autocompletion
...
Don't load password when autocompleting CLI commands
2023-10-23 20:06:53 +00:00
Michael Eischer
cd09ef4614
don't load password when autocompleting CLI commands
2023-10-23 21:44:22 +02:00
Michael Eischer
d399e32590
Merge pull request #4535 from MichaelEischer/fix-cat-command
...
Only show index loading progress bar if stdout is a terminal
2023-10-23 21:28:13 +02:00
Michael Eischer
54a4034ec0
Only show index progress bar if stdout is a terminal
...
This ensures that there is no interference when using restic for
scripting use cases.
2023-10-23 21:16:52 +02:00
rawtaz
138b7b3328
Merge pull request #4534 from MichaelEischer/polish-changelogs
...
Polish changelogs
2023-10-23 19:00:01 +00:00
Leo R. Lundgren
6d19e0260d
doc: Polish changelogs
2023-10-23 20:45:15 +02:00
Michael Eischer
85abceb99c
doc: update minimum required Go version
2023-10-23 19:02:20 +02:00
Michael Eischer
85c15e6fa3
polish changelogs
2023-10-23 19:01:45 +02:00
Michael Eischer
d6917c7e00
Merge pull request #4533 from MichaelEischer/update-dependencies
...
update dependencies
2023-10-23 19:01:03 +02:00
Michael Eischer
8c20301172
update dependencies
2023-10-23 18:48:13 +02:00
Michael Eischer
4b4f916bdc
Merge pull request #4532 from MichaelEischer/update-zstd
...
Update klauspost/compress to fix data corruption at maximum compression
2023-10-23 18:45:35 +02:00
Michael Eischer
9707956375
Merge pull request #4530 from MichaelEischer/repair-packs
...
Add `repair packs` command to simplify recovery from damaged pack
2023-10-23 18:44:42 +02:00
Michael Eischer
d1d4510974
repair packs: Add stub feature flag implementation
2023-10-23 18:36:28 +02:00
Michael Eischer
a28940ea29
check: Suggest usage of `restic repair packs` for corrupted blobs
...
For now, the guide is only shown if the blob content does not match its
hash. The main intended usage is to handle data corruption errors when
using maximum compression in restic 0.16.0
2023-10-23 18:36:28 +02:00
Michael Eischer
db26dc75e1
repair packs: add experimental command
...
This allows recovering a repository from several cases of damaged blobs.
2023-10-23 18:36:28 +02:00
Michael Eischer
efef38d0e8
drop support for Go 1.18
2023-10-23 18:30:38 +02:00
Michael Eischer
d00e72fed4
Update klauspost/compress to fix data corruption at maximum compression
2023-10-23 18:30:38 +02:00
Arash Farr
d15ffd9c92
retry: Do not retry Load() if file does not exist
2023-10-22 13:25:32 -05:00
Michael Eischer
62af0d769a
Merge pull request #4528 from adamantike/misc/run-tests-with-go-1.21
...
Run tests with Go 1.21
2023-10-22 16:00:27 +00:00
Michael Eischer
ae83a9002a
Merge pull request #4531 from MichaelEischer/silence-automaxprocs
...
Disable log output from automaxprocs library
2023-10-22 17:43:36 +02:00
Michael Eischer
ceff4af1ac
Disable log output from automaxprocs library
2023-10-22 17:09:46 +02:00
Michael Manganiello
b15ba553a4
Run tests with Go 1.21
2023-10-22 10:47:42 -03:00
Michael Eischer
46cb1df1bc
Merge pull request #4505 from MichaelEischer/verify-release-binaries
...
Add script to verify release binaries
2023-10-21 20:00:25 +02:00
Michael Eischer
bd3816fa14
CI: Ensure that github containers match the official binaries
...
The binaries accidentally included VCS information whereas binaries
built from the release tarball do not. For consistency remove the .git
directory before building the container on Github.
2023-10-21 19:44:59 +02:00
Michael Eischer
b2b0856908
Add helper script to verify release binaries
...
The script checks that the released binaries and the container binaries
can be reproduced.
2023-10-21 19:44:59 +02:00
Michael Eischer
7f05af02b9
Merge pull request #4480 from AgathaSorceress/add-rest-auth-env
...
Support reading basic auth credentials for REST server from environment variables
2023-10-21 17:41:08 +00:00
Michael Eischer
eabc177a42
Merge pull request #4442 from Refutable4890/copy-doc
...
Add a description of the resumption of `copy`
2023-10-21 17:38:15 +00:00
Michael Eischer
ab6defbace
Merge pull request #4519 from adamantike/backend/sftp/add-args-option
...
backend/sftp: Add sftp.args option
2023-10-21 17:37:46 +00:00
Michael Eischer
fe1f61570b
Merge pull request #4518 from adamantike/backend/b2/switch-to-official-library
...
backend/b2: Switch to official library `Backblaze/blazer`
2023-10-21 17:29:19 +00:00
Michael Eischer
baf9b54891
sftp: simplify documentation for sftp.args
2023-10-21 19:28:27 +02:00
Michael Eischer
6a4d6d5da4
sftp: test that Args and Command options cannot be set at the same time
2023-10-21 19:26:39 +02:00
Michael Manganiello
41f70f1f4f
backend/sftp: Add sftp.args option
...
Allow setting custom arguments for the `sftp` backend, by using the
`sftp.args` option. This is similar to the approach already implemented
in the `rclone` backend, to support new arguments without requiring
future code changes for each different SSH argument.
Closes #4241
2023-10-21 19:25:44 +02:00
Michael Eischer
6c7560e537
backend/b2: Switch to official library `Backblaze/blazer`
...
According to https://github.com/kurin/blazer Readme, the repository is
no longer maintained, and the official one at
https://github.com/Backblaze/blazer took over.
Closes #4148
2023-10-21 19:18:13 +02:00
Agatha V. Lovelace
0f97356b21
docs: add new REST server env variables
2023-10-20 13:04:58 +02:00
Agatha V. Lovelace
2089c54310
changelog: add unreleased entry
2023-10-20 13:04:58 +02:00
Agatha V. Lovelace
f1877e721e
feat: support reading REST credentials from env
2023-10-20 13:04:58 +02:00
Michael Eischer
17f2301cc2
Merge pull request #4514 from grembo/key-list-no-lock
...
Make `key list` command honor `--no-lock`
2023-10-19 20:50:56 +00:00
Michael Eischer
79deb99605
Merge pull request #4500 from MichaelEischer/simplify-issue-template
...
Simplify issue template
2023-10-19 20:48:36 +00:00
Michael Eischer
643180b415
simplify issue template
2023-10-19 22:16:38 +02:00
Michael Eischer
d27cfd10a9
Merge pull request #4487 from lbausch/tweak-restore-summary
...
Tweak restore summary
2023-10-19 20:14:14 +00:00
Michael Gmelin
34f3b13b7c
Make `key list` command honor `--no-lock`
...
Fixes #4513
2023-10-19 22:09:35 +02:00
Michael Eischer
495982232c
Merge pull request #4511 from dnnr/inode-in-json
...
Include inode in JSON output for find and ls
2023-10-17 21:06:34 +00:00
Michael Eischer
d173d1c532
Merge pull request #4486 from MichaelEischer/separate-large-blobs
...
repository: store oversized blobs in separate pack files
2023-10-17 23:03:52 +02:00
Michael Eischer
f955222750
Merge pull request #4502 from MichaelEischer/replace-ratelimit
...
backend/limiter: replace juju/ratelimit with x/time/rate
2023-10-17 23:03:17 +02:00
Michael Eischer
cb9cbe55d9
repository: store oversized blobs in separate pack files
...
Store oversized blobs in separate pack files as the blobs is large
enough to warrant its own pack file. This simplifies the garbage
collection of such blobs and keeps the cache smaller, as oversize (tree)
blobs only have to be downloaded if they are actually used.
2023-10-17 22:52:16 +02:00
Michael Eischer
f750aa8dfb
backend/limiter: replace juju/ratelimit with x/time/rate
2023-10-17 22:49:25 +02:00
Michael Eischer
c635e30e3f
Merge pull request #4485 from MichaelEischer/uber-automaxprocs
...
Automatically adjust GOMAXPROCS when run in a cpu-constrained container
2023-10-17 22:47:10 +02:00
Michael Eischer
f5d5e8fd0a
automatically adjust GOMAXPROCS when run in a cpu-constrained container
2023-10-17 20:31:57 +02:00
Daniel Danner
a858ab254b
Include inode in JSON output for find and ls
...
This introduces the inode attribute to the JSON output emitted for nodes
in `ls` and matches in `find`. There doesn't seem to be any discernible
reason to omit the inode and it can be useful in scripting scenarios.
2023-10-11 21:10:01 +02:00
Michael Eischer
4087c3aff7
Merge pull request #4498 from PeterDaveHello/ImproveDockerfile
...
Remove additional `--update` for apk in Dockerfile
2023-10-09 20:53:49 +00:00
Refutable4890
aa86c76aab
Add a description of the resumption of `copy`
2023-10-08 12:07:40 +08:00
Michael Eischer
76ef94d15c
Merge pull request #4419 from arjunajesh/issue#299
...
Progress Bar when loading index files
2023-10-01 18:06:35 +00:00
Michael Eischer
91aef00df3
check: add index loading progress bar
2023-10-01 19:55:29 +02:00
Michael Eischer
b0da0f152f
deduplicate index progress bar setup
2023-10-01 19:53:26 +02:00
Michael Eischer
75f6bd89ed
hide index loading progress for JSON output
2023-10-01 19:53:26 +02:00
Michael Eischer
3fd0ad7448
repository: list index files only once
2023-10-01 19:53:26 +02:00
arjunajesh
b6593ad7df
changelog
2023-10-01 19:53:26 +02:00
arjunajesh
ed65a7dbca
implement progress bar for index loading
2023-10-01 19:52:59 +02:00
Michael Eischer
eac1c4a8d0
Merge pull request #4471 from jalseth/master
...
docs(examples): Use restic binary from system path
2023-10-01 12:31:28 +00:00
Michael Eischer
f519454f33
Merge pull request #4493 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.33.0
...
build(deps): bump cloud.google.com/go/storage from 1.31.0 to 1.33.0
2023-10-01 12:09:43 +00:00
Michael Eischer
54ae8a0c40
Merge pull request #4492 from restic/dependabot/github_actions/docker/login-action-b4bedf8053341df3b5a9f9e0f2cf4e79e27360c6
...
build(deps): bump docker/login-action from cf8514a65188af1d4f94f8c28a7a4153af1088ce to b4bedf8053341df3b5a9f9e0f2cf4e79e27360c6
2023-10-01 12:08:10 +00:00
Michael Eischer
a36b5b6391
Merge pull request #4491 from restic/dependabot/github_actions/docker/setup-buildx-action-3
...
build(deps): bump docker/setup-buildx-action from 2 to 3
2023-10-01 12:03:48 +00:00
Michael Eischer
5e36e4da96
Merge pull request #4489 from restic/dependabot/github_actions/docker/setup-qemu-action-3
...
build(deps): bump docker/setup-qemu-action from 2 to 3
2023-10-01 12:03:19 +00:00
Michael Eischer
8ee08e5d09
Merge pull request #4490 from restic/dependabot/github_actions/actions/checkout-4
...
build(deps): bump actions/checkout from 3 to 4
2023-10-01 12:03:09 +00:00
Michael Eischer
9f9f736ec2
Merge pull request #4495 from restic/dependabot/go_modules/golang.org/x/oauth2-0.12.0
...
build(deps): bump golang.org/x/oauth2 from 0.11.0 to 0.12.0
2023-10-01 12:01:12 +00:00
Michael Eischer
af98c3ccbe
Merge pull request #4496 from restic/dependabot/go_modules/github.com/hashicorp/golang-lru/v2-2.0.7
...
build(deps): bump github.com/hashicorp/golang-lru/v2 from 2.0.6 to 2.0.7
2023-10-01 11:59:38 +00:00
Peter Dave Hello
6edfc73879
Remove additional `--update` for apk in Dockerfile
2023-10-01 18:00:15 +08:00
dependabot[bot]
d985ed27d1
build(deps): bump github.com/hashicorp/golang-lru/v2 from 2.0.6 to 2.0.7
...
Bumps [github.com/hashicorp/golang-lru/v2](https://github.com/hashicorp/golang-lru ) from 2.0.6 to 2.0.7.
- [Release notes](https://github.com/hashicorp/golang-lru/releases )
- [Commits](https://github.com/hashicorp/golang-lru/compare/v2.0.6...v2.0.7 )
---
updated-dependencies:
- dependency-name: github.com/hashicorp/golang-lru/v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:45:19 +00:00
dependabot[bot]
4278ec6553
build(deps): bump golang.org/x/oauth2 from 0.11.0 to 0.12.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.11.0 to 0.12.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.11.0...v0.12.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:45:07 +00:00
dependabot[bot]
15cb498c47
build(deps): bump cloud.google.com/go/storage from 1.31.0 to 1.33.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.31.0 to 1.33.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.31.0...pubsub/v1.33.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:44:54 +00:00
dependabot[bot]
59e217b003
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from cf8514a65188af1d4f94f8c28a7a4153af1088ce to b4bedf8053341df3b5a9f9e0f2cf4e79e27360c6.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](cf8514a651...b4bedf8053
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:24:15 +00:00
dependabot[bot]
3457f50c8c
build(deps): bump docker/setup-buildx-action from 2 to 3
...
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action ) from 2 to 3.
- [Release notes](https://github.com/docker/setup-buildx-action/releases )
- [Commits](https://github.com/docker/setup-buildx-action/compare/v2...v3 )
---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:24:10 +00:00
dependabot[bot]
bbe2f1ecf2
build(deps): bump actions/checkout from 3 to 4
...
Bumps [actions/checkout](https://github.com/actions/checkout ) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases )
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md )
- [Commits](https://github.com/actions/checkout/compare/v3...v4 )
---
updated-dependencies:
- dependency-name: actions/checkout
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:24:05 +00:00
dependabot[bot]
362917afb9
build(deps): bump docker/setup-qemu-action from 2 to 3
...
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action ) from 2 to 3.
- [Release notes](https://github.com/docker/setup-qemu-action/releases )
- [Commits](https://github.com/docker/setup-qemu-action/compare/v2...v3 )
---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-10-01 01:24:01 +00:00
Michael Eischer
b92ab458b0
Merge pull request #4459 from restic/dependabot/go_modules/github.com/go-ole/go-ole-1.3.0
...
build(deps): bump github.com/go-ole/go-ole from 1.2.6 to 1.3.0
2023-09-28 20:42:14 +00:00
Lorenz Bausch
2657217574
Tweak restore summary
2023-09-20 10:59:55 +02:00
James Alseth
02ab511c2f
docs(examples): Use restic binary from system path
...
Signed-off-by: James Alseth <james@jalseth.me>
2023-09-17 19:39:45 -07:00
Michael Eischer
6e586b64e4
Merge pull request #4450 from MichaelEischer/cleanup-master-index-test
...
master_index: fix inconsistent length blob length in test
2023-09-06 22:55:48 +02:00
Michael Eischer
fb4d458cce
Merge pull request #4452 from lbausch/docs/add-current-key-hint
...
docs: explain meaning of asterisk when managing repository keys
2023-09-06 20:28:33 +00:00
Michael Eischer
c7b5ddc012
Merge pull request #4451 from greatroar/bh-string
...
internal/restic: Return summary from BlobSet.String
2023-09-06 20:27:26 +00:00
Michael Eischer
3eb825e47c
Merge pull request #4464 from restic/dependabot/go_modules/golang.org/x/term-0.12.0
...
build(deps): bump golang.org/x/term from 0.11.0 to 0.12.0
2023-09-06 20:18:34 +00:00
dependabot[bot]
4d60011030
build(deps): bump golang.org/x/term from 0.11.0 to 0.12.0
...
Bumps [golang.org/x/term](https://github.com/golang/term ) from 0.11.0 to 0.12.0.
- [Commits](https://github.com/golang/term/compare/v0.11.0...v0.12.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/term
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-09-05 21:43:28 +00:00
Michael Eischer
507ed32469
Merge pull request #4457 from restic/dependabot/go_modules/google.golang.org/api-0.138.0
...
build(deps): bump google.golang.org/api from 0.134.0 to 0.138.0
2023-09-05 21:32:34 +00:00
dependabot[bot]
22fdfe1ffe
build(deps): bump github.com/go-ole/go-ole from 1.2.6 to 1.3.0
...
Bumps [github.com/go-ole/go-ole](https://github.com/go-ole/go-ole ) from 1.2.6 to 1.3.0.
- [Release notes](https://github.com/go-ole/go-ole/releases )
- [Changelog](https://github.com/go-ole/go-ole/blob/master/ChangeLog.md )
- [Commits](https://github.com/go-ole/go-ole/compare/v1.2.6...v1.3.0 )
---
updated-dependencies:
- dependency-name: github.com/go-ole/go-ole
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-09-05 21:17:03 +00:00
Michael Eischer
e05cd9abca
Merge pull request #4462 from restic/dependabot/github_actions/docker/login-action-cf8514a65188af1d4f94f8c28a7a4153af1088ce
...
build(deps): bump docker/login-action from a9794064588be971151ec5e7144cb535bcb56e36 to cf8514a65188af1d4f94f8c28a7a4153af1088ce
2023-09-05 21:07:57 +00:00
Michael Eischer
ea55ca5303
Merge pull request #4458 from restic/dependabot/go_modules/github.com/hashicorp/golang-lru/v2-2.0.6
...
build(deps): bump github.com/hashicorp/golang-lru/v2 from 2.0.4 to 2.0.6
2023-09-05 21:05:45 +00:00
Michael Eischer
df53f4782b
Merge pull request #4453 from lbausch/docs/fix-table
...
docs: fix malformed table
2023-09-05 20:37:30 +00:00
Michael Eischer
35055adbc4
Merge pull request #4454 from lbausch/tweak-readme
...
README: add Mastodon link
2023-09-05 20:36:32 +00:00
dependabot[bot]
bd9eb528c0
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from a9794064588be971151ec5e7144cb535bcb56e36 to cf8514a65188af1d4f94f8c28a7a4153af1088ce.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](a979406458...cf8514a651
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-09-01 01:45:03 +00:00
dependabot[bot]
02032f3109
build(deps): bump github.com/hashicorp/golang-lru/v2 from 2.0.4 to 2.0.6
...
Bumps [github.com/hashicorp/golang-lru/v2](https://github.com/hashicorp/golang-lru ) from 2.0.4 to 2.0.6.
- [Release notes](https://github.com/hashicorp/golang-lru/releases )
- [Commits](https://github.com/hashicorp/golang-lru/compare/v2.0.4...v2.0.6 )
---
updated-dependencies:
- dependency-name: github.com/hashicorp/golang-lru/v2
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-09-01 01:41:13 +00:00
dependabot[bot]
cfff1367c1
build(deps): bump google.golang.org/api from 0.134.0 to 0.138.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.134.0 to 0.138.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.134.0...v0.138.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-09-01 01:40:56 +00:00
Lorenz Bausch
353265a329
README: add Mastodon link
2023-08-27 10:52:12 +02:00
Lorenz Bausch
84a1170dee
docs: remove trailing whitespace
2023-08-27 10:49:34 +02:00
Lorenz Bausch
0b4d9c9a51
docs: add missing whitespace to avoid malformed table warning
2023-08-27 10:46:55 +02:00
Lorenz Bausch
5422a7daa5
docs: explain meaning of asterisk when managing repository keys
2023-08-27 10:40:55 +02:00
greatroar
691c01963b
internal/restic: Return summary from BlobSet.String
...
Fixes #4449 : error messages from prune were too long to fit in scroll
buffers.
2023-08-25 21:41:30 +02:00
Michael Eischer
2bec99dc6f
master_index: fix inconsistent length blob length in test
...
Two blobs with the same hash must always have the same content length.
2023-08-19 20:04:25 +02:00
Michael Eischer
e60c5b2d7f
Merge pull request #4436 from MichaelEischer/add-snapshot-subfolder-syntax-to-help-text
...
Add snapshot subfolder syntax in help output of diff/dump/restore
2023-08-13 19:36:38 +02:00
Michael Eischer
a04964bb86
Merge pull request #4446 from 2000yeshu/feat-cat
...
Check for arguments before opening remote connection in cat command
2023-08-13 17:25:37 +00:00
Michael Eischer
fe54912a46
cat: extract parameter validation and add a test
2023-08-13 19:17:46 +02:00
Michael Eischer
feb6abb7bb
Merge pull request #4394 from greatroar/acl
...
dump: Rewrite Linux ACL handling
2023-08-13 17:01:31 +00:00
greatroar
aaf5254e26
dump: Rewrite Linux ACL handling
...
The old version was taken from an MPL-licensed library. This is a
cleanroom implementation. The code is shorter and it's now explicit that
only Linux ACLs are supported.
2023-08-13 13:37:11 +02:00
Yakul Garg
705556f134
Check for arguments before opening remote connection in cat command
...
Signed-off-by: Yakul Garg <2000yeshu@gmail.com>
2023-08-12 23:54:40 +05:30
Michael Eischer
c23eebc225
Merge pull request #4428 from MichaelEischer/fix-restore-help
...
restore: Fix help text for `--iinclude` and `--iexclude`
2023-08-12 20:21:22 +02:00
Refutable4890
e09f6f540f
Add a description of the resumption of `copy`
2023-08-08 12:51:40 +08:00
Michael Eischer
0fca028491
restore: Fix help text for `--iinclude` and `--iexclude`
2023-08-05 18:21:03 +02:00
Michael Eischer
57a08291f5
Merge pull request #4429 from restic/dependabot/github_actions/docker/login-action-a9794064588be971151ec5e7144cb535bcb56e36
...
build(deps): bump docker/login-action from 65b78e6e13532edd9afa3aa52ac7964289d1a9c1 to a9794064588be971151ec5e7144cb535bcb56e36
2023-08-05 13:41:08 +00:00
Michael Eischer
2117ce4cfb
Merge pull request #4434 from lbausch/docs-fix-wording
...
docs: fix `percent_done` description
2023-08-05 13:39:02 +00:00
Michael Eischer
3a478bc522
unify snapshotID spelling in command parameters
2023-08-05 12:35:45 +02:00
Michael Eischer
9a7704fa2b
diff/dump/restore: describe snapshot:subfolder syntax in help text
2023-08-05 12:35:08 +02:00
Lorenz Bausch
9a69f44de2
docs: fix `percent_done` description
2023-08-04 16:15:18 +02:00
dependabot[bot]
7a6339180b
build(deps): bump docker/login-action
...
Bumps [docker/login-action](https://github.com/docker/login-action ) from 65b78e6e13532edd9afa3aa52ac7964289d1a9c1 to a9794064588be971151ec5e7144cb535bcb56e36.
- [Release notes](https://github.com/docker/login-action/releases )
- [Commits](65b78e6e13...a979406458
)
---
updated-dependencies:
- dependency-name: docker/login-action
dependency-type: direct:production
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-08-01 01:19:51 +00:00
Alexander Neumann
82e6e28781
Set development version for 0.16.0
2023-07-31 20:24:29 +02:00
Alexander Neumann
eb33e564c9
Add version for 0.16.0
2023-07-31 20:23:24 +02:00
Alexander Neumann
62680af734
Update manpages and auto-completion
2023-07-31 20:23:24 +02:00
Alexander Neumann
68460fd3d1
Generate CHANGELOG.md for 0.16.0
2023-07-31 20:22:50 +02:00
Alexander Neumann
9459328d8d
Prepare changelog for 0.16.0
2023-07-31 20:22:50 +02:00
rawtaz
9d71990c26
Merge pull request #4424 from MichaelEischer/tweak-changelog
...
Tweak changelog entries
2023-07-29 11:29:30 +00:00
rawtaz
9219b6a0ef
Merge pull request #4427 from MichaelEischer/changelog-template
...
Extend style guide in changelog template
2023-07-29 11:15:17 +00:00
Michael Eischer
9a0ec05145
Extend style guide in changelog template
2023-07-29 12:55:56 +02:00
Michael Eischer
2a5b6af2e9
extend gc tweak changelog
2023-07-29 12:51:05 +02:00
Leo R. Lundgren
51e4e5ef82
doc: Polish changelog
2023-07-29 12:51:04 +02:00
Michael Eischer
e998314088
tweak changelog entries
2023-07-29 12:50:01 +02:00
Michael Eischer
be03c1a457
Merge pull request #4422 from MichaelEischer/document-non-utf-symlinks
...
Document `linktarget_raw` field for non utf symlinks
2023-07-28 23:14:08 +02:00
Michael Eischer
95050117eb
Merge pull request #4426 from MichaelEischer/consistent-forget-policy
...
forget: replace `--keep-* -1` with `--keep-* unlimited`
2023-07-28 19:06:00 +00:00
Michael Eischer
21a3a41b69
Merge pull request #4423 from MichaelEischer/update-dependencies
...
Update dependencies
2023-07-28 19:25:02 +02:00
Michael Eischer
3f919f2371
forget: simplify test
2023-07-28 19:21:40 +02:00
Michael Eischer
50b43fbac0
forget: replace `--keep-* -1` with `--keep-* unlimited`
...
This ensures consistency with the `prune --max-unused unlimited` option.
2023-07-28 19:21:40 +02:00
Michael Eischer
f689e2638d
reenable aix builds
2023-07-27 21:27:31 +02:00
Michael Eischer
f9ef2b1e44
update all dependencies
2023-07-27 21:26:44 +02:00
Michael Eischer
b87a37f318
doc: Fix typos
2023-07-25 20:14:56 +02:00
Michael Eischer
647ebf352a
doc: Describe linktarget_raw field
2023-07-25 20:14:47 +02:00
Michael Eischer
62d3ef4a93
Merge pull request #4417 from MichaelEischer/fix-flaky-list-cancel-test
...
backend: Fix unreliable TestListCancel/Timeout test
2023-07-23 18:04:11 +02:00
Michael Eischer
6c5fc32967
Merge pull request #3225 from wojas/restic-cmd-more-help
...
Extend help output with short overview
2023-07-23 15:43:03 +00:00
Michael Eischer
ae13cf15c1
doc: move quickstart to docs
2023-07-23 17:32:37 +02:00
Michael Eischer
1fe1ec40a2
Merge pull request #4409 from arjunajesh/issue#1495
...
snapshots sorted by timestamp in restic find
2023-07-23 11:00:41 +00:00
arjunajesh
32f5ee6f4e
snapshots sorted by timestamp
2023-07-23 12:49:23 +02:00
Konrad Wojas
5bd8a6d7eb
Extend help output with short overview
...
The list of subcommands can be confusing for first time users. This
adds a short overview of the most important commands to help get people
started.
2023-07-23 12:01:40 +02:00
Michael Eischer
4a33370072
backend: Fix unreliable TestListCancel/Timeout test
...
The test uses `WithTimeout` to create a context that cancels the List
operation after a given delay. Several backends internally use a derived
child context created using WithCancel.
The cancellation of a context first closes the done channel of the
context (here: the `WithTimeout` context) and _afterwards_ propagates
the cancellation to child contexts (here: the `WithCancel` context).
Therefor if the List implementation uses a child context, then it may
take a moment until that context is also cancelled. Thus give the
context cancellation a moment to propagate.
2023-07-23 11:47:56 +02:00
Michael Eischer
98fb56baa6
Merge pull request #3802 from MichaelEischer/support-non-utf-symlinks
...
Support non utf symlink targets
2023-07-23 00:15:01 +02:00
Michael Eischer
f12bbd9229
restic: check that Node.LinkTarget can handle non-utf8 characters
2023-07-23 00:05:14 +02:00
Michael Eischer
6adb629608
Add support for non-utf8 symlink targets
2023-07-23 00:05:14 +02:00
Michael Eischer
25ff9fa893
Merge pull request #4334 from MichaelEischer/snapshot-subtree-syntax
...
Add support for snapshot:path syntax
2023-07-22 23:59:26 +02:00
Michael Eischer
bdaec8fdb8
Merge pull request #4374 from MichaelEischer/try-refresh-stale-locks
...
Try to refresh stale locks
2023-07-22 23:50:52 +02:00
Michael Eischer
55b440b520
tweak snapshot:subfolder docs
2023-07-22 23:50:17 +02:00
Michael Eischer
3f63b53090
lock: Shrink critical section in RefreshStaleLock
...
A cleaning up after a failed lock refresh attempt does not require a
mutex.
2023-07-22 23:29:14 +02:00
Michael Eischer
76253b2a20
Hopefully fix flaky TestLockWaitCancel test
2023-07-22 23:23:17 +02:00
Michael Eischer
20e82d1fcf
lock: add changelog for stale lock refresh
2023-07-22 23:23:17 +02:00
Michael Eischer
96eada3d5f
backend/sema: add test for freeze functionality
2023-07-22 23:23:17 +02:00
Michael Eischer
b8f4267a36
lock: add more debug information to flaky test
2023-07-22 23:23:17 +02:00
Michael Eischer
1f6883a05c
lock: test stale lock refresh
2023-07-22 23:23:17 +02:00
Michael Eischer
8154f6a77a
Ensure consistent naming for `<snapshot>:<subfolder>` syntax
2023-07-22 19:55:57 +02:00
Michael Eischer
184b7616ba
document snapshot:subfolder syntax
2023-07-22 19:55:57 +02:00
Michael Eischer
67f237b4f3
add changelog for snapshot:path syntax
2023-07-22 19:55:57 +02:00
Michael Eischer
090f9d6237
restic: Cleanup and simplify TestCreateSnapshot
2023-07-22 19:55:57 +02:00
Michael Eischer
321cc35cde
restic: add test for FindTreeDirectory
2023-07-22 19:55:57 +02:00
Michael Eischer
a97915642c
restic: Ensure snapshots created by TestCreateSnapshot are valid
...
All nodes in a tree must be sorted by node name. Otherwise functionality
like searching for a specific tree node will break.
2023-07-22 19:55:57 +02:00
Michael Eischer
c64d81063e
test snapshot filtering with snapshot:path syntax
2023-07-22 19:55:57 +02:00
Michael Eischer
233b841ad9
Add cat tree snapshot:subpath command
...
This command prints the tree for the given path.
2023-07-22 19:55:57 +02:00
Michael Eischer
85860e6e97
Add support for snapshot:subpath syntax
...
This snapshot specification syntax is supported by the cat, diff, dump,
ls and restore command.
2023-07-22 19:55:57 +02:00
Michael Eischer
2dd6769429
lock: Fix possible deadlock during refresh of stale lock
...
A delayed lock refresh could send a signal on the `refreshed` channel
while the `monitorLockRefresh` goroutine waits for a reply to its
`refreshLockRequest`. As the channels are unbuffered, this resulted in a
deadlock.
2023-07-22 19:55:44 +02:00
Michael Eischer
5d9b0d894e
lock: add unit test for RefreshStaleLock
2023-07-22 19:55:44 +02:00
Michael Eischer
d4bf7a3cb1
restic: cleanup lock test
2023-07-22 19:55:44 +02:00
Michael Eischer
24c8a33da9
restic: reduce sleeps in lock tests
2023-07-22 19:55:44 +02:00
Michael Eischer
399f8e84a1
lock: use mem backend for locking tests
...
This hopefully fixes the timing related test failures which appear to be
caused by very slow fsync calls.
2023-07-22 19:55:44 +02:00
Michael Eischer
6a436d731d
lock: rename test helper function
2023-07-22 19:55:44 +02:00
Michael Eischer
b2fcbc21cb
lock: rework stale lock refresh to avoid data race
2023-07-22 19:55:44 +02:00
Michael Eischer
05e5e29a8c
backup: don't pretend to lock repo for dry run
...
The dry run did not actually create lock files, but still ran all other
parts of the lock processing.
2023-07-22 19:55:44 +02:00
Michael Eischer
f490288738
lock: freeze backend operations while refreshing stale lock
...
Freeze new backend operations while trying to refresh a stale lock.
2023-07-22 19:55:44 +02:00
Michael Eischer
51718ec561
restic: generic implemention of AsBackend
2023-07-22 19:55:44 +02:00
Michael Eischer
11eb88a2ea
lock: cleanup
2023-07-22 19:55:44 +02:00
Michael Eischer
bee3231ed4
lock: try refreshing of stale locks
...
A stale lock may be refreshed if it continues to exist until after a
replacement lock has been created. This ensures that a repository was
not unlocked in the meantime.
2023-07-22 19:55:44 +02:00
Michael Eischer
60d8066568
Merge pull request #4407 from MichaelEischer/simpler-required-ci
...
CI: Add check that verifies the results of other checks
2023-07-22 19:55:06 +02:00
Michael Eischer
08dea911bd
CI: Add check that verifies the results of other checks
...
To properly use merge queues, it is necessary to mark certain checks as
"required" in the branch protection rules. However, this doesn't scale
with matrix jobs and would require manual tweaks of the CI jobs change.
The new analyze job simplifies this by allowing the branch protection
rule just check for that job. All further dependencies are then
configured within the CI config.
2023-07-16 14:13:41 +02:00
Michael Eischer
47206a6579
Merge pull request #4351 from kenny-y-dev/human-readable-ls
...
Add --human-readable to ls and find output
2023-07-16 11:44:41 +00:00
Michael Eischer
594689db32
Merge pull request #4400 from MichaelEischer/rest-handle-missing-dirs
...
rest: Ignore missing directories
2023-07-14 23:29:37 +02:00
Michael Eischer
5705326bb8
add changelog
2023-07-14 22:39:38 +02:00
Michael Eischer
978ebaac49
rest: use http status code constants
2023-07-14 22:39:38 +02:00
Michael Eischer
307aeb6849
rest: Don't return error if listing non-existent directory
...
When transferring a repository from S3 to, for example, a local disk
then all empty folders will be missing.
When saving files, the missing intermediate folders are created
automatically. Therefore, missing directories can be ignored by the
`List()` operation.
2023-07-14 22:39:38 +02:00
Michael Eischer
9cd85d5956
Merge pull request #4029 from ekarlso/add-azure-workload-identity-support
...
add azure workload identity support
2023-07-09 19:06:59 +00:00
Michael Eischer
c9f506925c
azure: Document additional auth options
2023-07-09 20:56:21 +02:00
Michael Eischer
e990d3d483
azure: Support authentication using managed / workload identity
...
See https://github.com/Azure/azure-sdk-for-go/tree/sdk/azidentity/v1.3.0/sdk/azidentity
2023-07-09 14:47:03 +02:00
Michael Eischer
7042190807
Merge pull request #4403 from MichaelEischer/fix-certificate-env-handling
...
Fix certificate env handling
2023-07-09 14:46:38 +02:00
Michael Eischer
c158741e2e
CI: add minimal CLI test
...
Just create a repository and run a minimal backup.
2023-07-09 14:29:43 +02:00
Michael Eischer
89fbd39e59
Don't print stacktrace on invalid cacert option
2023-07-09 14:23:54 +02:00
Michael Eischer
1ce599d2ae
Fix handling of empty cacert environment variable
...
This resulted in a "empty filename for root certificate supplied" error.
2023-07-09 14:23:54 +02:00
Michael Eischer
789fec3da7
Merge pull request #4402 from MichaelEischer/merge-queue-setup
...
CI: Enable missing CI tests for Github merge queue
2023-07-09 13:57:10 +02:00
Michael Eischer
8a120c8800
CI: Enable missing CI tests for Github merge queue
2023-07-08 23:51:39 +02:00
Michael Eischer
41395e83c5
Merge pull request #2740 from metalsp0rk/document-json
...
Add JSON documentation
2023-07-08 21:34:02 +00:00
Michael Eischer
2a193195b0
Merge pull request #4364 from MichaelEischer/multiplatform-docker
...
Setup multiplatform docker builds
2023-07-08 23:31:58 +02:00
Michael Eischer
229c7b24a4
doc: add program_version field of snapshot JSON output
2023-07-08 23:31:20 +02:00
Michael Eischer
b34ce57dd4
doc: describe JSON output of restore command
2023-07-08 23:31:20 +02:00
Michael Eischer
1ce839228e
doc: Expand JSON documentation
2023-07-08 23:31:20 +02:00
Kyle Brennan
fb1170c1d6
doc tweaks
2023-07-08 23:21:41 +02:00
Kyle Brennan
e457fe22bc
Documentation improvement
2023-07-08 23:21:41 +02:00
Kyle Brennan
39299e36ef
add json documentation
2023-07-08 23:21:41 +02:00
Michael Eischer
0512f292e8
Merge pull request #4361 from MichaelEischer/stats-repo-debug-info
...
stats: Add `debug` mode to collect repository statistics
2023-07-08 23:20:23 +02:00
Michael Eischer
3a93e28605
CI: Remove .dockerignore to ensure reproducible builds
...
Since go 1.18, built binaries also include VCS information such as the
built commit. This information is also included in the official
binaries. To ensure that the Docker container recreates the same
binaries, the .git folder must also be transferred into the container.
Thus, remove the .dockerignore file.
The copied files must also be owned by the current user within the
container, as git refuses to work otherwise.
2023-07-08 23:16:14 +02:00
Michael Eischer
6b82cce1bd
add changelog for multiplatform containers
2023-07-08 23:16:14 +02:00
Michael Eischer
31e07cecbb
docker: update to Go 1.20 for custom container builds
2023-07-08 23:05:39 +02:00
Michael Eischer
c181b51360
doc: Update docker build process
2023-07-08 23:05:38 +02:00
Michael Eischer
ccd19b7e88
CI: run cloud backend tests only on restic/restic
2023-07-08 23:05:02 +02:00
Michael Eischer
b0987ff570
CI: only build containers on restic/restic
2023-07-08 23:05:02 +02:00
Michael Eischer
eff3124f15
CI: Setup automatic container builds for ghcr.io
...
Containers are built for new tags and pushes to the master branch.
2023-07-08 23:05:02 +02:00
Michael Eischer
43fa051546
Directly build restic binary in release Docker container
2023-07-08 23:05:02 +02:00
Michael Eischer
2293835242
Release multi-platform docker containers
2023-07-08 23:05:02 +02:00
Michael Eischer
0fcb6c7f94
ask for debug statistics in resource usage issues
2023-07-08 23:04:47 +02:00
Michael Eischer
325fa916b5
stats: Add debug mode to collect repository statistics
2023-07-08 23:04:47 +02:00
Michael Eischer
170e495334
Merge pull request #4401 from MichaelEischer/fix-windows-build
...
fuse: fix windows build
2023-07-08 23:01:22 +02:00
Michael Eischer
9d44682e3e
fuse: fix windows build
2023-07-08 22:40:22 +02:00
Michael Eischer
4d43509423
Merge pull request #4379 from chenxiaolong/symlink_xattrs
...
Add support for extended attributes on symlinks
2023-07-08 16:56:51 +00:00
Michael Eischer
ea9ad77e05
restic: refactor node test
2023-07-08 18:54:56 +02:00
Michael Eischer
cc84884d2e
restic: basic xattr test for files/dirs
2023-07-08 18:49:21 +02:00
Michael Eischer
4a5ae2ba84
restic: test NodeFromFileInfo for symlinks
2023-07-08 18:18:13 +02:00
Michael Eischer
1f1e50f49e
fuse: add test for symlink xattr
2023-07-08 18:02:17 +02:00
Michael Eischer
f3c3b0f377
fuse: deduplicate xattr code
2023-07-08 17:41:45 +02:00
Michael Eischer
7e2be9e081
Merge pull request #4384 from arjunajesh/issue#1926
...
certificates can be passed through env vars
2023-07-08 07:46:31 +00:00
Michael Eischer
cc3c218baf
small cleanups for certificate environment variables
2023-07-08 09:44:59 +02:00
arjunajesh
c1578a2035
certificates can be passed through env vars
2023-07-08 09:44:59 +02:00
Michael Eischer
678b983300
Merge pull request #4360 from Refutable4890/master
...
Add an description of `check` temporary cache directory location
2023-07-07 21:51:52 +00:00
Michael Eischer
1b3870dc43
Merge pull request #4378 from MichaelEischer/add-version-to-snapshot
...
Add program version to snapshot
2023-07-07 23:50:44 +02:00
Michael Eischer
ef40aee2bd
Merge pull request #4373 from MichaelEischer/performance-issue-help
...
document how to analyze performance / memory usage issues
2023-07-07 23:50:27 +02:00
Michael Eischer
3a32c4e59f
document how to analyze performance / memory usage issues
2023-07-07 23:38:53 +02:00
Michael Eischer
e703e89e9b
add changelog for program version
2023-07-07 23:36:57 +02:00
Michael Eischer
389f6ee74c
backup: add minimal test for program versioni
2023-07-07 23:33:23 +02:00
Michael Eischer
bbac74b172
add program version to snapshot
2023-07-07 23:27:10 +02:00
Michael Eischer
a280b7364e
Merge pull request #4395 from greatroar/parsebytes
...
cmd, ui: Move size parsing code and make it more robust
2023-07-07 21:24:47 +00:00
Michael Eischer
825651a135
Merge pull request #4387 from arjunajesh/issue#2468
...
azure domain can be overridden with env variable
2023-07-07 21:17:01 +00:00
Michael Eischer
e36d17a6f8
azure: tweak documentation for endpoint suffix
2023-07-07 23:10:26 +02:00
arjunajesh
068b115abc
added azure domain parameter
2023-07-07 23:10:26 +02:00
greatroar
41a5bf357f
cmd, ui: Move size parsing code and make it more robust
2023-07-02 21:44:50 +02:00
Michael Eischer
f96896a9c0
Merge pull request #4383 from LouisMT/add-docker-hostname
...
Add hostname flag to Docker docs
2023-07-01 18:58:43 +02:00
dependabot[bot]
2ab9a3b9c3
Merge pull request #4389 from restic/dependabot/go_modules/google.golang.org/api-0.129.0
2023-07-01 16:57:18 +00:00
Louis Matthijssen
dbe2eef80c
Add hostname flag to Docker docs
...
Fixes #4380
2023-07-01 18:46:31 +02:00
dependabot[bot]
6e7c6674ad
build(deps): bump google.golang.org/api from 0.116.0 to 0.129.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.116.0 to 0.129.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.116.0...v0.129.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-01 16:11:44 +00:00
Michael Eischer
7fe830ee1e
Merge pull request #4392 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.6.1
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.6.0 to 1.6.1
2023-07-01 18:10:47 +02:00
Michael Eischer
a07c7166ba
Merge pull request #4390 from restic/dependabot/go_modules/golang.org/x/oauth2-0.9.0
...
build(deps): bump golang.org/x/oauth2 from 0.8.0 to 0.9.0
2023-07-01 18:10:38 +02:00
dependabot[bot]
d2f8f9de23
build(deps): bump golang.org/x/oauth2 from 0.8.0 to 0.9.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.8.0 to 0.9.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.8.0...v0.9.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-01 15:32:55 +00:00
dependabot[bot]
d5fe5107c8
Merge pull request #4391 from restic/dependabot/go_modules/golang.org/x/sys-0.9.0
2023-07-01 15:31:42 +00:00
dependabot[bot]
f08ba1a005
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.6.0 to 1.6.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.6.0...sdk/azcore/v1.6.1 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-01 01:23:19 +00:00
dependabot[bot]
70fb554854
build(deps): bump golang.org/x/sys from 0.8.0 to 0.9.0
...
Bumps [golang.org/x/sys](https://github.com/golang/sys ) from 0.8.0 to 0.9.0.
- [Commits](https://github.com/golang/sys/compare/v0.8.0...v0.9.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sys
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-07-01 01:23:05 +00:00
Andrew Gunnerson
8c02ebb029
Add support for extended attributes on symlinks
...
Linux allows the use of non-`user.` extended attributes on symlinks. One
of the main users of this functionality is SELinux's `security.selinux`
xattr for storing a path's label. By storing symlink xattrs, restic is
now suitable for backing up the root filesystem on Linux distributions
that use SELinux.
This commit adds support for symlink xattrs when backing up data,
restoring data, and mounting snapshots via a fuse mount. All calls to
the xattr library have been updated to the use `L` variants of the
various functions, which always operate on the path given, without
following symlinks.
Fixes : #4375
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-06-19 14:37:31 -04:00
Michael Eischer
8dcb0c4a9d
doc: improve description of caching behavior of the check command
2023-06-17 22:18:50 +02:00
Michael Eischer
74ca82a6f8
Merge pull request #4362 from restic/unified-backend-factory
...
Unified backend initialization
2023-06-17 15:52:37 +02:00
Michael Eischer
9cbc2502c6
Merge pull request #4366 from gautammenghani/master
...
Issue #3624 : Preserve oldest snapshot when keep-* values are not sati…
2023-06-17 15:41:17 +02:00
Michael Eischer
93038ed8f4
s3: restore retries for minio tests
2023-06-17 15:25:08 +02:00
Gautam Menghani
8da5a6649b
Preserve oldest snapshot when keep-within* does not collect enough
2023-06-17 15:17:20 +02:00
Michael Eischer
3888c21a27
reword changelog
2023-06-17 15:16:58 +02:00
Michael Eischer
1257c2c075
forget: Add comments to snapshot policy
2023-06-17 15:16:58 +02:00
Gautam Menghani
182b9796e4
Issue #3624 : Preserve oldest snapshot when keep-* values are not satisfied
2023-06-17 15:16:58 +02:00
Michael Eischer
cbf87fbdb3
init: don't include password in debug log
2023-06-17 15:15:58 +02:00
Michael Eischer
b5511e8e4c
Fix linter warnings
2023-06-17 15:15:58 +02:00
Michael Eischer
50e0d5e6b5
backend: Hardcode backend scheme in Factory
...
Our ParseConfig implementations always expect a specific scheme, thus no
other scheme would work.
2023-06-17 15:15:58 +02:00
Michael Eischer
705ad51bcc
backend: check that StripPassword can be called
2023-06-17 15:15:58 +02:00
Michael Eischer
13a8b5822f
backend: Adjust tests to use the Factory to instantiate the backend
...
This drastically reduces the amount of duplicated test code.
2023-06-17 15:15:58 +02:00
Michael Eischer
3d3bb88745
b2: remove duplicate check for config file during repository creation
...
No other backend implements that check. The check that a repository is
not yet initialized is handled by the Repository later on.
2023-06-17 15:15:58 +02:00
Michael Eischer
3a3cf608f5
b2/s3: Move config validation from ApplyEnvironment to Open/Create
...
Conceptually the backend configuration should be validated when creating
or opening the backend, but not when filling in information from
environment variables into the configuration.
2023-06-17 15:15:58 +02:00
Michael Eischer
19ac12d95b
location: make genericBackendFactory private
2023-06-17 15:15:58 +02:00
Michael Eischer
3325a7c862
location: extract backend specific part of StripPassword
...
The tests for the rest backend now reside there.
2023-06-17 15:15:58 +02:00
Michael Eischer
555be49a79
location: Make ParseConfig-test backend agnostic
...
The backend specific parts of the test are now directly handled by the
respective backend. Duplicate tests were removed.
2023-06-17 15:15:58 +02:00
Michael Eischer
9aa9e0d1ec
local/sftp: move limiter setup into backend
2023-06-17 15:15:58 +02:00
Michael Eischer
7d12c29286
backend: Unify backend construction using factory and registry
...
This unified construction removes most backend-specific code from
global.go. The backend registry will also enable integration tests to
use custom backends if necessary.
2023-06-17 15:15:57 +02:00
Michael Eischer
56836364a4
backend: pass context into every backend constructor
2023-06-17 15:15:57 +02:00
Michael Eischer
4df77e9f26
Merge pull request #4314 from MichaelEischer/restore-json
...
Add JSON support for the restore progress bar
2023-06-16 23:53:17 +02:00
Michael Eischer
2545c84321
Merge pull request #4365 from MichaelEischer/ci-rework-build-tests
...
CI: rework build tests
2023-06-16 23:29:39 +02:00
Michael Eischer
191c47d30e
Merge pull request #4353 from MichaelEischer/tune-gc
...
Tune Go garbage collector
2023-06-16 23:24:39 +02:00
Michael Eischer
dd1ef13c1c
Merge pull request #4352 from MichaelEischer/pointerless-index
...
index: optimize go GC performance
2023-06-16 23:23:14 +02:00
Michael Eischer
4d5ee987a7
add changelog about missing AIX builds
2023-06-16 23:19:36 +02:00
Michael Eischer
b2ed42cec4
index: add basic hat test
2023-06-16 23:12:30 +02:00
Michael Eischer
61042a77a4
building on aix is currently not possible
2023-06-09 13:35:35 +02:00
Michael Eischer
4e9e2c3229
CI: Use build-release-binaries to run the cross-compilation tests
...
gox silently ignored linux/mips and aix/ppc64. This change also removes
the duplicate platform list.
2023-06-09 13:27:57 +02:00
Michael Eischer
faec0ff816
build-release-binaries: support building a subset of all platforms
2023-06-09 13:27:33 +02:00
Michael Eischer
07d1f8047e
restore: More descriptive field names for the JSON output
2023-06-09 12:07:55 +02:00
Michael Eischer
b2b0760eb0
restore: add test for json output
2023-06-09 12:03:33 +02:00
Michael Eischer
cf16239058
restore: print output via termStatus
2023-06-09 12:03:33 +02:00
Michael Eischer
1531eab746
mention restore json support in changelog
2023-06-09 12:03:33 +02:00
Michael Eischer
d54176ce5d
restore: add basic json progress
2023-06-09 12:03:33 +02:00
Michael Eischer
a9aff885d6
restore: reorganize progress bar code
...
The structure is now much more similar to that of the backup command.
2023-06-09 12:03:33 +02:00
Michael Eischer
bb20078641
restore: pass termStatus to restore in tests
2023-06-09 12:03:33 +02:00
Michael Eischer
237f32c651
Merge pull request #3261 from DRON-666/prealloc-packs
...
Prevent local backend file fragmentation by file preallocation.
2023-06-09 11:50:46 +02:00
DRON-666
74e4656850
Update changelog
2023-06-09 11:49:06 +03:00
DRON-666
c37d587f81
Use PreallocateFile in local backend
2023-06-09 11:49:06 +03:00
DRON-666
ffc6b3d887
Move PreallocateFile to fs package
2023-06-09 11:49:06 +03:00
Michael Eischer
88c63a029c
Merge pull request #3563 from matta/improve-eta
...
Improve the ETA displayed during backup
2023-06-08 20:49:19 +02:00
Michael Eischer
0b908bb1fb
Address review comments
2023-06-08 20:24:21 +02:00
Matt Armstrong
0372c7ef04
Improve the ETA displayed during backup
...
The ETA restic displays was based on a rate computed across the entire
backup operation. Often restic can progress at uneven rates. In the worst
case, restic progresses over most of the backup at a very high rate and
then finds new data to back up. The displayed ETA is then unrealistic and
never adapts.
Restic now estimates the transfer rate based on a sliding window, with the
goal of adapting to observed changes in rate. To avoid wild changes in the
estimate, several heuristics are used to keep the sliding window wide
enough to be relatively stable.
2023-06-08 20:05:56 +02:00
Michael Eischer
9464c63550
Make formatNode test timezone independent
...
formatNode formats the timestamp according to the current time zone. Pin
the local timezone to UTC to ensure the test works everywhere.
2023-06-08 19:18:30 +02:00
Michael Eischer
6ebf2dd235
Reduce duplicate code in test for fomatNode
2023-06-08 19:16:16 +02:00
Michael Eischer
5f153109ba
Refactor formatNode
2023-06-08 19:12:49 +02:00
Michael Eischer
2beaa74892
tweak changelog
2023-06-08 19:12:26 +02:00
Michael Eischer
55c21846b1
Revert "index: remove redundant storage of indexmap size"
...
This reverts commit f1c388c623
.
For an uninitialized indexmap the returned size was `-1` which is
unexpected and could cause problems.
2023-06-08 18:08:46 +02:00
Michael Eischer
0f80b6a137
add changelog for gc tuning
2023-06-08 18:02:46 +02:00
Michael Eischer
e14ccb1142
Merge pull request #4298 from restic/backend-parseconfig-cleanup
...
Unified and slightly type-safer backend config parsing
2023-06-08 12:02:27 +02:00
Michael Eischer
609367195a
backend: Fix test compilation with Go 1.18 and 1.19
2023-06-07 22:31:15 +02:00
Michael Eischer
18eb1d3ab0
backend: test cleanup
2023-06-07 22:31:15 +02:00
Michael Eischer
32a6b66267
backend: add standardized Config.ApplyEnvironment
...
This removes the backend specific special cases while parsing the
configuration in `global.go`.
2023-06-07 22:31:15 +02:00
Michael Eischer
f903db492c
backend: let ParseConfig return a Config pointer
...
In order to change the backend initialization in `global.go` to be able
to generically call cfg.ApplyEnvironment() for supported backends, the
`interface{}` returned by `ParseConfig` must contain a pointer to the
configuration.
An alternative would be to use reflection to convert the type from
`interface{}(Config)` to `interface{}(*Config)` (from value to pointer
type). However, this would just complicate the type mess further.
2023-06-07 22:31:15 +02:00
Michael Eischer
25a0be7f26
backend: fix linter warnings
2023-06-07 22:31:15 +02:00
Michael Eischer
a27b7f1370
backend: use generic instead of any type for test suite
2023-06-07 22:31:14 +02:00
Michael Eischer
fa361dbfbd
backend: use generic implementation for ParseConfig tests
2023-06-07 22:30:35 +02:00
Michael Eischer
5260d38980
backend: let ParseConfig return concrete type
2023-06-07 22:30:35 +02:00
Michael Eischer
2f7b4ceae1
backend: Move environment based configuration into backend
2023-06-07 22:30:35 +02:00
Refutable4890
aea7538936
Add an description of `check` temporary cache directory location
2023-06-07 22:35:33 +08:00
Michael Eischer
49a6a4f5bf
Merge pull request #4356 from MichaelEischer/fix-unlock-hint
...
lock: fix missing hint to unlock command if repository is locked
2023-06-06 21:03:06 +02:00
Michael Eischer
237d00000e
lock: fix missing hint to unlock command if repository is locked
2023-06-02 23:16:49 +02:00
dependabot[bot]
0c727f6ad1
Merge pull request #4355 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.56
2023-06-02 20:51:26 +00:00
Michael Eischer
4e7d3efad4
Merge pull request #4345 from DanielGibson/patch-1
...
restic restore docs: Mention that restic restore is fastest
2023-06-02 22:44:38 +02:00
Michael Eischer
17446da5fd
Merge pull request #4343 from greatroar/cache
...
cache: Restructure New to remove redundant operations
2023-06-02 22:41:17 +02:00
dependabot[bot]
a3cee840d2
build(deps): bump github.com/minio/minio-go/v7 from 7.0.52 to 7.0.56
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.52 to 7.0.56.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.52...v7.0.56 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 20:35:42 +00:00
dependabot[bot]
c76f3a1e27
Merge pull request #4348 from restic/dependabot/go_modules/github.com/spf13/cobra-1.7.0
2023-06-02 20:34:38 +00:00
dependabot[bot]
1e9714088d
Merge pull request #4347 from restic/dependabot/go_modules/golang.org/x/sync-0.2.0
2023-06-02 20:32:27 +00:00
Michael Eischer
58e3f5955c
Merge pull request #4350 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.6.0
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.4.0 to 1.6.0
2023-06-02 22:26:55 +02:00
dependabot[bot]
dd8d8b1ae0
build(deps): bump github.com/spf13/cobra from 1.6.1 to 1.7.0
...
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra ) from 1.6.1 to 1.7.0.
- [Release notes](https://github.com/spf13/cobra/releases )
- [Commits](https://github.com/spf13/cobra/compare/v1.6.1...v1.7.0 )
---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 20:22:39 +00:00
dependabot[bot]
e1ac0f0e0c
build(deps): bump golang.org/x/sync from 0.1.0 to 0.2.0
...
Bumps [golang.org/x/sync](https://github.com/golang/sync ) from 0.1.0 to 0.2.0.
- [Commits](https://github.com/golang/sync/compare/v0.1.0...v0.2.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/sync
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-02 20:22:36 +00:00
Michael Eischer
a88d90b8e5
Merge pull request #4346 from restic/dependabot/go_modules/golang.org/x/oauth2-0.8.0
...
build(deps): bump golang.org/x/oauth2 from 0.7.0 to 0.8.0
2023-06-02 22:21:22 +02:00
Michael Eischer
2fcb3947df
prune: trigger GC after prune planning
2023-06-02 21:57:40 +02:00
Michael Eischer
eef0ee7a85
repository: trigger GC after loading the index
...
Loading the index requires some scratch space, thus make sure that this
memory does not factor into the targeted gc memory usage limit.
2023-06-02 21:56:14 +02:00
Michael Eischer
f1b73c9301
Reduce GOGC to 50
...
The index used by restic consumes a major part of the total memory
usage. This leads to an unnecessarily large amount of memory that
contains ephemeral objects that are only used for a short time.
2023-06-02 21:51:50 +02:00
Michael Eischer
ac1dfc99bb
index: fix blocklist size
2023-06-02 19:39:12 +02:00
Kenny Y
098de3554c
Add pull request link
2023-06-01 21:30:51 -04:00
Kenny Y
8812dcd56a
Add --human-readable to ls and find output
...
Modifies format module to add options for human readable storage size formatting, using size parsing already in ui/format.
Cmd flag --human-readable added to ls and find commands.
Additional option added to formatNode to support printing size in regular or new human readable format
2023-06-01 21:18:18 -04:00
dependabot[bot]
379282299a
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.4.0 to 1.6.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.4.0...sdk/azcore/v1.6.0 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-01 01:59:16 +00:00
dependabot[bot]
0c796dbd9b
build(deps): bump golang.org/x/oauth2 from 0.7.0 to 0.8.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.7.0 to 0.8.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.7.0...v0.8.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-06-01 01:58:29 +00:00
Daniel Gibson
f9dded83b3
doc/050_restore.rst: Mention that restic restore is fastest
...
Especially with lots of files (like when backing up git repos),
restic mount + cp -r can be really slow.
2023-05-31 22:44:36 +02:00
Michael Eischer
88a10a368f
Merge pull request #4342 from greatroar/errors
...
cmd: Don't check for errno == 0
2023-05-31 22:17:07 +02:00
Michael Eischer
9a7056a479
index: implement indexmap.grow() without random access
2023-05-30 20:13:33 +02:00
Michael Eischer
fc05e35a08
index: let indexmap.Each iterate in allocation order
...
Iterating through the indexmap according to the bucket order has the
problem that all indexEntries are accessed in random order which is
rather cache inefficient.
As we already keep a list of all allocated blocks, just iterate through
it. This allows iterating through a batch of indexEntries without random
memory accesses. In addition, the packID will likely remain similar
across multiple blobs as all blobs of a pack file are added as a single
batch.
2023-05-30 20:12:36 +02:00
Michael Eischer
f1c388c623
index: remove redundant storage of indexmap size
2023-05-30 20:11:53 +02:00
Michael Eischer
12141afbad
index: Allow inlining of HAT
2023-05-30 20:11:14 +02:00
Michael Eischer
fed33295c3
index: store indexEntries in hashed array tree
...
This data structure reduces the wasted memory to O(sqrt(n)). The
top-layer of the hashed array tree (HAT) also has a size of O(sqrt(n)),
which makes it cache efficient. The top-layer should be small enough to
easily fit into the CPU cache and thus only adds little overhead
compared to directly accessing an index entry via a pointer.
2023-05-29 00:24:15 +02:00
Michael Eischer
b217f38ee7
index: Remove pointers from within indexentrys
...
The indexEntry objects are now allocated in a separate array. References
to an indexEntry are now stored as array indices. This has the benefit
of allowing the garbage collector to ignore the indexEntry objects as
these do not contain pointers and are part of a single large allocation.
2023-05-29 00:24:15 +02:00
Michael Eischer
0c1240360d
index: add garbage collection benchmark
...
Allocates an index and repeatedly triggers the GC.
2023-05-29 00:23:04 +02:00
Michael Eischer
ffca602315
repository: Fix panic in benchmarkLoadIndex
2023-05-28 23:55:47 +02:00
greatroar
da419be43c
cache: Restructure New to remove redundant operations
...
New and its helpers used to create the cache directories several times
over. They now only do so once. The added test ensures that the cache is
produced in a consistent state when parts are deleted.
2023-05-27 10:32:08 +02:00
greatroar
8f1ca8fabe
cmd: Don't check for errno == 0
...
Maybe term.Restore's predecessor could return this, but the current
implementation never does.
2023-05-27 08:52:37 +02:00
Michael Eischer
f74dad2afb
Merge pull request #4340 from restic/least-privilege-ci
...
CI: least privilege for workflows
2023-05-26 20:29:05 +02:00
Michael Eischer
3e287afdbf
Merge pull request #4226 from Sohalt/gcs-region
...
gs: support other regions than us
2023-05-26 20:28:52 +02:00
Michael Eischer
06894484a1
shorten changelog title
2023-05-26 20:05:49 +02:00
Michael Eischer
6e5b42d5c4
CI: least privilege for workflows
2023-05-26 19:58:02 +02:00
sohalt
ed5b2c2c9b
gs: support other regions than us
2023-05-26 19:54:42 +02:00
Michael Eischer
13c32b0fbe
Merge pull request #4328 from MichaelEischer/cleanup-integration-tests
...
Cleanup integration tests
2023-05-26 19:49:16 +02:00
Michael Eischer
9747cef338
fix linter warnings
2023-05-26 19:38:44 +02:00
Michael Eischer
8e913e6d3a
repair index: always read Quiet flags from GlobalOptions passed as parameter
2023-05-26 19:38:44 +02:00
Michael Eischer
b93459cbb0
repair snapshots: use local copy of globalOptions to open repository
2023-05-26 19:38:44 +02:00
Michael Eischer
692f81ede8
cleanup prune integration test
2023-05-26 19:38:44 +02:00
Michael Eischer
7a268e4aba
always access stdout/stderr via globalOptions
2023-05-26 19:38:44 +02:00
Michael Eischer
4b3a0b4104
read JSON/Quiet flag from the passed in globalOptions
2023-05-26 19:38:44 +02:00
Michael Eischer
cebce52c16
test: add helper to capture stdout for integration tests
2023-05-26 19:38:44 +02:00
Michael Eischer
675a49a95b
Restructure integration tests
...
The tests are now split into individual files for each command. The
separation isn't perfect as many tests make use of multiple commands. In
particular `init`, `backup`, `check` and `list` are used by a larger
number of test cases.
Most tests now reside in files name cmd_<name>_integration_test.go. This
provides a certain indication which commands have significant test
coverage.
2023-05-26 19:38:43 +02:00
Michael Eischer
e2dba9f5c7
test: cleanup a some check calls
2023-05-26 19:34:57 +02:00
Michael Eischer
06fd6b54d7
test: print log output if testRunCheck fails
2023-05-26 19:34:57 +02:00
Michael Eischer
419e6f26b1
tests: Simplify checks that a specific number of snapshots exists
2023-05-26 19:34:57 +02:00
Michael Eischer
c3212ab6a6
test: use standard logging methods from testing for the test helpers
...
Use the logging methods from testing.TB to make use of tb.Helper(). This
allows the tests to log the filename and line number in which the test
helper was called. Previously the test helper was logged which is rarely
useful.
2023-05-26 19:34:57 +02:00
Michael Eischer
658aa4c0f7
Merge pull request #4339 from greatroar/ioctl
...
backend, termstatus: fix ioctl calls for 64-bit big-endian platforms
2023-05-26 19:34:14 +02:00
Michael Eischer
998cf5a7f8
Merge pull request #4333 from MichaelEischer/staticcheck
...
Update golangci-lint and resolve all errors
2023-05-26 19:34:00 +02:00
Michael Eischer
7eec91f841
add changelog
2023-05-26 19:23:51 +02:00
greatroar
51dc80be5b
backend: Use IoctlSetPointerInt for tcsetpgrp
...
This function casts its argument to int32 before passing it to the
system call, so that big-endian CPUs read the lower rather than the
upper 32 bits of the pid.
This also gets rid of the last import of "unsafe" in the Unix build.
I changed syscall to x/sys/unix while I was at it, to remove one more
import line. The constants and types there are aliases for their syscall
counterparts.
2023-05-25 17:31:51 +02:00
greatroar
ddbc0c1b37
termstatus: Fix IsProcessBackground for 64-bit big-endian Linux
...
Fixes #4223 . Verified with QEMU on linux/amd64.
2023-05-25 17:20:42 +02:00
Michael Eischer
ecbf8e055c
CI: upgrade golangci-lint and check full codebase
2023-05-18 21:17:53 +02:00
Michael Eischer
16ba237d8b
azure: Fix missing rate limiting
2023-05-18 21:17:53 +02:00
Michael Eischer
a466e945d9
stats: pass StatsOptions via parameter
2023-05-18 21:17:53 +02:00
Michael Eischer
03b9764bce
init/generate: check that no parameters are passed
2023-05-18 21:17:53 +02:00
Michael Eischer
22c9276719
rest: remove unused parameter
2023-05-18 21:17:53 +02:00
Michael Eischer
1e33b285c1
lock: remove unused parameter
2023-05-18 21:17:53 +02:00
Michael Eischer
c05f96e6b9
repair index: remove no longer used paramter
2023-05-18 21:17:53 +02:00
Michael Eischer
94752b7ee2
backup: Remove unused parameters from rejectFunc collection
2023-05-18 21:17:53 +02:00
Michael Eischer
0058745881
test: use parameter instead of hardcoded constant
2023-05-18 21:17:53 +02:00
Michael Eischer
a719d10e22
termstatus: remove unused parameter from helpers
2023-05-18 21:17:53 +02:00
Michael Eischer
b0a01ae68a
ui/backup: remove unused parameter from ProgressPrinter interface
2023-05-18 21:17:53 +02:00
Michael Eischer
472bf5184f
Replace lots of unused parameters with _
...
The parameters are required by the implemented function signature or interface.
2023-05-18 21:17:53 +02:00
Michael Eischer
d1a5ec7839
Rename unused testing parameter to _
...
The parameter is an additional marker that the test helper must only be
used for tests.
2023-05-18 21:17:53 +02:00
Michael Eischer
1514593f22
Remove unused context or testing parameters
2023-05-18 21:17:53 +02:00
Michael Eischer
5e4e268bdc
Use _ as parameter name for unused Context
...
The context is required by the implemented interface.
2023-05-18 21:15:45 +02:00
Michael Eischer
3252f60df5
Ignore deprecated imports for poly1305 and openpgp
2023-05-18 21:15:45 +02:00
Michael Eischer
2fa8b96843
Don't shadow builtins
2023-05-18 21:15:45 +02:00
Michael Eischer
7a01bd3b67
Remove unnecessary else block
2023-05-18 21:15:45 +02:00
Michael Eischer
319087c056
Remove redundant if ...; err != nil; return pattern
2023-05-18 21:15:45 +02:00
Michael Eischer
6ed73ed408
CI: remove deprecated linters
2023-05-18 21:15:45 +02:00
Michael Eischer
c832a492ac
fuse: use syscall errnos directly to fix deprecations
2023-05-18 21:15:45 +02:00
Michael Eischer
e01baeabba
Use either test or rtest to refer to internal test helpers
...
A single test file should not use both names.
2023-05-18 21:15:45 +02:00
Michael Eischer
bfc9c6c971
Merge pull request #4331 from MichaelEischer/fix-mount-failures
...
Hopefully fix `TestMount` failures
2023-05-18 20:22:57 +02:00
Michael Eischer
5773b86d02
repository: Push all usage of errors.Fatal out of the package
...
As the `Fatal` error type only includes a string, it becomes impossible
to inspect the contained error. This is for a example a problem for the
fuse implementation, which must be able to detect context.Canceled
errors.
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2023-05-18 17:27:41 +02:00
Michael Eischer
a013014c24
backup: return normal error if --group-by cannot be parsed
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2023-05-18 17:23:31 +02:00
Michael Eischer
f9850b79b5
rest/sftp: Remove redundant fatal from error message
...
This caused restic to exit with error messages like
`Fatal: parsing repository location failed: Fatal: sftp path [...]`
`Fatal: create repository at rest:http://localhost:12345/ failed: Fatal: config file already exists`
2023-05-18 17:22:26 +02:00
Michael Eischer
2f518b7241
Merge pull request #4304 from MichaelEischer/unlimited-lock-refresh
...
lock: Do not limit backend concurrency for lock files
2023-05-13 21:40:29 +02:00
Michael Eischer
49be202cb0
Merge pull request #4315 from MichaelEischer/restore-hardlink-progress
...
restore: Correctly account for hardlinks in progress bar
2023-05-13 21:37:40 +02:00
Michael Eischer
19ebc1b786
restore: Add basic test for progress bar accounting of hardlinks
2023-05-08 20:51:49 +02:00
Michael Eischer
23a122a901
restore: count files in the same way as the stats command
2023-05-08 20:51:49 +02:00
Michael Eischer
e77002f841
restore: correctly count hardlinks in progress bar
...
For hardlinked files, only the first instance of that file increases the
amount of bytes to restore. All later instances only increase the file
count but not the restore size.
2023-05-08 20:51:49 +02:00
Michael Eischer
d05f6211d1
lock: Do not limit backend concurrency for lock files
...
restic must be able to refresh lock files in time. However, large
uploads over slow connections can cause the lock refresh to be stuck
behind the large uploads and thus time out.
2023-05-08 19:04:46 +02:00
Michael Eischer
ee3c55ea3d
Merge pull request #2876 from aawsome/new-repair-command
...
Add repair command
2023-05-05 23:22:24 +02:00
Michael Eischer
db046c0acc
Merge pull request #4318 from MichaelEischer/status-output-truncation
...
Fix status output truncation
2023-05-05 23:09:59 +02:00
Michael Eischer
3e6a26e2e9
Merge pull request #4317 from MichaelEischer/debug-test-mount-failures
...
mount: enable debug logging for the flaky TestMount test
2023-05-05 23:08:03 +02:00
greatroar
65c5e511a1
ui/termstatus: Add test for Terminal.SetStatus
2023-05-05 22:51:41 +02:00
Michael Eischer
6d10c655a0
termstatus: test status line sanitization
2023-05-05 22:51:41 +02:00
Michael Eischer
bb40e49e75
ui/termstatus: Fix truncation of status output
...
The last line was not truncated as expected
2023-05-05 22:45:52 +02:00
Michael Eischer
fefe15d7a1
Merge pull request #4316 from MichaelEischer/fix-check-modified-data
...
Fix flaky TestCheckerModifiedData test
2023-05-05 22:38:28 +02:00
Michael Eischer
78e5aa6d30
repair snapshots: add basic tests
2023-05-04 23:00:46 +02:00
Michael Eischer
1cb11ad8ad
mount: enable debug logging for the flaky TestMount test
...
The test case fails from time to time with an Input/Output error while
trying to access the snapshots directory.
2023-05-01 18:03:17 +02:00
Michael Eischer
90a663c94f
Merge pull request #4308 from greatroar/tar-header
...
dump: Report filename with tar.ErrFieldTooLong
2023-05-01 17:30:31 +02:00
Michael Eischer
88a7231217
report snapshot id if loading failed
2023-05-01 17:24:13 +02:00
Michael Eischer
c0627dc80d
check: Fix flaky TestCheckerModifiedData
...
The test had a 4% chance of not modified the data read from the
repository, in which case the test would fail. Change the data
manipulation to just modified each read operation.
2023-05-01 17:18:19 +02:00
Michael Eischer
e71367e6b9
repair snapshots: update changelog
2023-05-01 16:06:17 +02:00
Michael Eischer
5aa37acdaa
repair snapshots: cleanup command help
2023-05-01 16:06:17 +02:00
Michael Eischer
9c64a95df8
doc: rewrite troubleshooting section
2023-05-01 16:06:17 +02:00
Michael Eischer
7c8dd61e8c
repair snapshots: cleanup warnings
2023-05-01 15:22:30 +02:00
Michael Eischer
f6cc10578d
repair snapshots: Always sanitize file nodes
...
If the node for a file is intact, this is a no-op.
2023-05-01 15:22:18 +02:00
Michael Eischer
4ce87a7f64
repair snapshots: port to filterAndReplaceSnapshot
...
The previous approach of rewriting all snapshots first, then flushing
the repository data and finally removing old snapshots has the downside
that an interrupted command execution leaves behind broken snapshots as
not all new data is already flushed.
2023-05-01 15:22:03 +02:00
Michael Eischer
e17ee40a31
repair snapshots: Port to use walker.TreeRewriter
2023-05-01 15:20:36 +02:00
Michael Eischer
1bd1f3008d
walker: extend TreeRewriter to support snapshot repairing
...
This adds support for caching already rewritten trees, handling of load
errors and disabling the check that the serialization doesn't lead to
data loss.
2023-05-01 15:20:24 +02:00
Michael Eischer
38dac78180
walker: restructure FilterTree into TreeRewriter
...
The more generic RewriteNode callback replaces the SelectByName and
PrintExclude functions. The main part of this change is a preparation to
allow using the TreeRewriter for the `repair snapshots` command.
2023-05-01 15:20:12 +02:00
Michael Eischer
bc2399fbd9
walker: recurse into directory based on node type
...
A broken directory might also not have a subtree.
2023-05-01 15:20:00 +02:00
Michael Eischer
1a9705fc95
walker: Simplify change detection in FilterTree
...
Now the rewritten tree is always serialized which makes sure that we
don't accidentally miss any relevant changes.
2023-05-01 15:19:48 +02:00
Michael Eischer
8c4caf09a8
repair snapshots: Do not rename repaired files
...
The files in a tree must be sorted in lexical order. However, this
cannot be guaranteed when appending a filename suffix. For two files
file, file.rep
where "file" is broken, this would result in
file.repaired, file.rep
which is no longer sorted.
In addition, adding a filename suffix is also prone to filename
collisions which would require a rather complex search for a
collision-free name in order to work reliably.
2023-05-01 15:19:36 +02:00
Michael Eischer
375189488c
rewrite: prepare for code sharing with rewrite snapshots
2023-05-01 15:19:24 +02:00
Michael Eischer
903651c719
repair snapshots: partially synchronize code with rewrite command
...
Simplify CLI options:
* Rename "DeleteSnapshots" to "Forget"
* Replace "AddTag" and "Append" with hardcoded values
Change output and snapshot modifications to be more in line with the
"rewrite" command.
2023-05-01 15:19:11 +02:00
Michael Eischer
118d599d0a
Rename 'rebuild-index' to 'repair index'
...
The old name still works, but is deprecated.
2023-05-01 15:16:44 +02:00
Michael Eischer
db459eda21
move to subcommand
2023-05-01 15:15:48 +02:00
Michael Eischer
a14a63cd29
modernize code
2023-05-01 15:12:09 +02:00
Alexander Weiss
947f0c345e
correct typos
2023-05-01 14:56:42 +02:00
Alexander Weiss
d23a2e1925
better error handling and correct nil tree behavior
2023-05-01 14:56:42 +02:00
Alexander Weiss
08ae708b3b
make linter happy
2023-05-01 14:56:42 +02:00
Alexander Weiss
99a05d5ab2
Update troubleshooting documentation
2023-05-01 14:56:42 +02:00
Alexander Weiss
6557f36f61
Add changelog and docu for #2876
2023-05-01 14:56:42 +02:00
Alexander Weiss
5f58797ba7
Add repair command
2023-05-01 14:56:42 +02:00
Alexander Weiss
9cef6b4c69
Add troubleshooting documentation
2023-05-01 14:56:42 +02:00
Michael Eischer
8a78a042db
Merge pull request #4310 from restic/dependabot/go_modules/github.com/klauspost/compress-1.16.5
...
build(deps): bump github.com/klauspost/compress from 1.16.0 to 1.16.5
2023-05-01 14:35:57 +02:00
Michael Eischer
b491af2b57
Merge pull request #4311 from restic/dependabot/go_modules/golang.org/x/crypto-0.8.0
...
build(deps): bump golang.org/x/crypto from 0.7.0 to 0.8.0
2023-05-01 14:35:37 +02:00
dependabot[bot]
d747a9c401
build(deps): bump golang.org/x/crypto from 0.7.0 to 0.8.0
...
Bumps [golang.org/x/crypto](https://github.com/golang/crypto ) from 0.7.0 to 0.8.0.
- [Release notes](https://github.com/golang/crypto/releases )
- [Commits](https://github.com/golang/crypto/compare/v0.7.0...v0.8.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/crypto
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-01 11:15:38 +00:00
dependabot[bot]
888c1ae63a
build(deps): bump github.com/klauspost/compress from 1.16.0 to 1.16.5
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.16.0 to 1.16.5.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.16.0...v1.16.5 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-01 11:07:29 +00:00
Michael Eischer
5eeb257c95
Merge pull request #4312 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.52
...
build(deps): bump github.com/minio/minio-go/v7 from 7.0.50 to 7.0.52
2023-05-01 13:00:48 +02:00
Michael Eischer
ae6729cf89
Merge pull request #4309 from restic/dependabot/go_modules/golang.org/x/oauth2-0.7.0
...
build(deps): bump golang.org/x/oauth2 from 0.6.0 to 0.7.0
2023-05-01 12:58:55 +02:00
dependabot[bot]
6abd494915
build(deps): bump github.com/minio/minio-go/v7 from 7.0.50 to 7.0.52
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.50 to 7.0.52.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.50...v7.0.52 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-01 01:59:50 +00:00
dependabot[bot]
7147a54ceb
build(deps): bump golang.org/x/oauth2 from 0.6.0 to 0.7.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/oauth2/releases )
- [Commits](https://github.com/golang/oauth2/compare/v0.6.0...v0.7.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-05-01 01:59:10 +00:00
Michael Eischer
8d971172c4
Merge pull request #4306 from MichaelEischer/document-restic-compression-env
...
Document that the compression mode can be set via $RESTIC_COMPRESSION
2023-04-30 18:26:10 +02:00
Michael Eischer
37d0e323eb
doc: update help output in manual_rest.rst
2023-04-30 16:12:29 +02:00
Michael Eischer
face5bd7f7
Document that the compression mode can be set via $RESTIC_COMPRESSION
2023-04-30 16:11:53 +02:00
Michael Eischer
1daf928a77
Merge pull request #4305 from MichaelEischer/stracktrace-for-windows
...
Print stacktrace in SIGINT handler if RESTIC_DEBUG_STACKTRACE_SIGINT set
2023-04-30 16:08:58 +02:00
Michael Eischer
37bab08181
Merge pull request #4302 from MichaelEischer/logging-cleanup
...
Tweak debug logs
2023-04-30 16:08:08 +02:00
Michael Eischer
6dc2324d2e
Merge pull request #4301 from MichaelEischer/less-restore-memory
...
restore: slightly reduce memory usage while restoring files
2023-04-30 16:07:43 +02:00
Michael Eischer
850cd9aace
Merge pull request #4299 from MichaelEischer/update-scripting-docs
...
doc: recommend `cat config` to check if a repository exists
2023-04-30 16:04:59 +02:00
greatroar
b50ff04cf3
dump: Report filename with tar.ErrFieldTooLong
...
Updates #4307 .
2023-04-27 11:45:41 +02:00
Michael Eischer
c8641f4479
Merge branch 'patch-release'
2023-04-24 20:44:38 +02:00
Alexander Neumann
ac7ac0cb97
Set development version for 0.15.2
2023-04-24 20:28:37 +02:00
Alexander Neumann
be8be3397c
Add version for 0.15.2
2023-04-24 20:28:37 +02:00
Alexander Neumann
db6b4f8912
Update manpages and auto-completion
2023-04-24 20:28:37 +02:00
Alexander Neumann
1f3f042f32
Generate CHANGELOG.md for 0.15.2
2023-04-24 20:28:24 +02:00
Alexander Neumann
0aaa4e6cbe
Prepare changelog for 0.15.2
2023-04-24 20:28:24 +02:00
Leo R. Lundgren
0bac935dac
doc: Polish changelogs
2023-04-23 22:10:04 +02:00
Michael Eischer
306a29980a
Print stacktrace in SIGINT handler if RESTIC_DEBUG_STACKTRACE_SIGINT set
...
The builtin mechanism to capture a stacktrace in Go is to send a SIGQUIT
to the running process. However, this mechanism is not avaiable on
Windows. Thus, tweak the SIGINT handler to dump a stacktrace if the
environment variable `RESTIC_DEBUG_STACKTRACE_SIGINT` is set.
2023-04-23 15:50:40 +02:00
Michael Eischer
1e6e9f9bd0
tweak changelogs
2023-04-23 12:46:55 +02:00
greatroar
f342db7666
ui/termstatus: Quote funny filenames
...
Fixes #2260 , #4191 .
2023-04-23 12:45:40 +02:00
Michael Eischer
41cc320145
Merge pull request #4166 from MichaelEischer/cache-mkdirall
...
cache: Create missing intermediate directories when caching a file
2023-04-23 11:44:18 +02:00
Michael Eischer
cdb0fb9c06
tweak debug logs
2023-04-23 11:38:06 +02:00
Michael Eischer
94cbc6392d
restore: slightly reduce memory usage while restoring files
...
The information which target files are contained in a pack file is no
longer necessary after processing a pack.
2023-04-23 11:33:21 +02:00
Michael Eischer
78a1757e5a
Cancel current command if cache becomes unusable
...
If the cache suddenly disappears, the current command will now fail.
2023-04-23 11:31:15 +02:00
Michael Eischer
22562d2132
Merge pull request #4300 from MichaelEischer/less-flaky-tests
...
Increase timeouts for lock refresh tests
2023-04-23 11:24:59 +02:00
Michael Eischer
51d823348d
Merge pull request #4286 from MichaelEischer/backend-cleanup-logging
...
Normalize backend logging and connection limiting
2023-04-23 11:24:26 +02:00
Michael Eischer
831f593b87
backend/sema: Add tests
2023-04-22 13:20:20 +02:00
Michael Eischer
179e11c2ae
Increase timeouts for lock refresh tests
...
When saving files to the local backend, in some cases the used fsync
calls are slow enough to cause the tests to time out. Thus, increase the
test timeouts as a stopgap measure until we can use the mem backend for
these tests.
2023-04-22 12:45:59 +02:00
Michael Eischer
ebba233a3a
backend/sema: rename constructor to NewBackend
2023-04-22 12:32:57 +02:00
rawtaz
8479390d7c
Merge pull request #3939 from Fabien-jrt/resticprofile
...
doc: Add resticprofile to scheduling section
2023-04-22 01:24:34 +02:00
Fabien-Jrt
756f43d5f9
doc: Add resticprofile to scheduling section
2023-04-22 01:04:48 +02:00
Michael Eischer
affd04c125
doc: recommend `cat config` to check if a repository exists
...
This will be much faster if a large number of snapshots exists.
2023-04-21 22:56:31 +02:00
Michael Eischer
f27750e270
backend/sema: rename type to connectionLimitedBackend
2023-04-21 22:53:09 +02:00
Michael Eischer
1dd873b706
Merge pull request #4293 from MichaelEischer/fix-groups-by
...
Fix parent snapshot selection for relative paths
2023-04-21 22:47:03 +02:00
Michael Eischer
7a60d9e54f
Merge pull request #4288 from MichaelEischer/log-warnings-to-debug-log
...
Add warnings via Warnf to the debug log
2023-04-21 22:39:00 +02:00
Michael Eischer
3001dd8c2b
Add test to verify that the backup parent is correctly selected
2023-04-21 22:35:02 +02:00
Michael Eischer
4503aea0df
Merge pull request #4296 from MichaelEischer/rewrite-reorder-output
...
rewrite: log snapshot saved before removal of the old snapshot
2023-04-19 22:00:32 +02:00
Michael Eischer
09cddb8927
rewrite: log snapshot saved before removal of the old snapshot
...
The snapshot was already saved before removing the old snapshot. Only
the log messages were printed in the wrong order.
2023-04-17 21:00:45 +02:00
Michael Eischer
913eab3361
Merge pull request #4234 from thndrbrrr/forget-opts-neg1-means-forever-issue-2565
...
restic forget --keep-* options will interpret -1 as "forever"
2023-04-14 23:18:47 +02:00
Michael Eischer
a9c7c12276
Merge pull request #4265 from HeikoSchlittermann/chg/progress-to-stderr
...
generate: do not write progress to STDOUT
2023-04-14 23:18:23 +02:00
Michael Eischer
85eef232e6
Merge pull request #4232 from thndrbrrr/copy-chunker-params-verbose-msg
...
init: Add --copy-chunker-params verbose msg
2023-04-14 23:17:56 +02:00
Michael Eischer
07a44a88f2
Fix snapshot filtering for relative paths in the backup command
...
The snapshot filtering internally converts relative paths to absolute
ones to ensure that the parent snapshots selection works for backups of
relative paths.
2023-04-14 22:58:34 +02:00
Michael Eischer
48e065d971
Sync dependency upgrades from master
...
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob still uses v0.5.1
as upgrading it would increase the minimum Go version on Solaris to
1.20.
2023-04-14 22:58:34 +02:00
Michael Eischer
a06d927dce
Merge pull request #4291 from greatroar/widechars
...
ui/termstatus: Optimize Truncate
2023-04-14 22:48:34 +02:00
Michael Eischer
fd3ed9e2f4
Merge pull request #4192 from greatroar/quote
...
ui/termstatus: Quote funny filenames
2023-04-14 22:39:09 +02:00
Michael Eischer
6042df075f
migrations: Fix S3 backend detection
2023-04-14 22:32:16 +02:00
Michael Eischer
c934c99d41
gs: replace usage of context.Background()
2023-04-14 22:32:15 +02:00
Michael Eischer
616926d2c1
gs: use IsNotExist to check error
2023-04-14 22:32:15 +02:00
Michael Eischer
05abc6d6f5
backend: deduplicate implementation of Delete() method
2023-04-14 22:32:15 +02:00
Michael Eischer
45244fdf68
backend: remove parameter validation tests
...
These parameter validations have been factored out into
SemaphoreBackend.
2023-04-14 22:32:15 +02:00
Michael Eischer
803640ba4b
backend: remove a few unnecessary debug logs
2023-04-14 22:32:15 +02:00
Michael Eischer
8e1e3844aa
backend: factor out connection limiting and parameter validation
...
The SemaphoreBackend now uniformly enforces the limit of concurrent
backend operations. In addition, it unifies the parameter validation.
The List() methods no longer uses a semaphore. Restic already never runs
multiple list operations in parallel.
By managing the semaphore in a wrapper backend, the sections that hold a
semaphore token grow slightly. However, the main bottleneck is IO, so
this shouldn't make much of a difference.
The key insight that enables the SemaphoreBackend is that all of the
complex semaphore handling in `openReader()` still happens within the
original call to `Load()`. Thus, getting and releasing the semaphore
tokens can be refactored to happen directly in `Load()`. This eliminates
the need for wrapping the reader in `openReader()` to release the token.
2023-04-14 22:32:15 +02:00
Michael Eischer
8b5ab5b59f
dryrun: fix outdated comments
2023-04-14 22:32:15 +02:00
Michael Eischer
4703473ec5
backend: extract most debug logs into logger backend
2023-04-14 22:32:15 +02:00
Michael Eischer
8bfc2519d7
backend: Deduplicate sanity checks for parameters of Load() method
...
The check is now handled by backend.DefaultLoad. This also guarantees
consistent behavior across all backends.
2023-04-14 22:32:15 +02:00
Michael Eischer
ba16904eed
backup: Add test to verify parent snapshot selection for relative paths
2023-04-14 22:21:43 +02:00
Michael Eischer
2841a87cc6
Fix snapshot filtering for relative paths in the backup command
...
The snapshot filtering internally converts relative paths to absolute
ones to ensure that the parent snapshots selection works for backups of
relative paths.
2023-04-14 21:53:55 +02:00
Michael Eischer
fab4a8a4d2
Properly initialize the --group-by option for backup tests
2023-04-14 21:53:01 +02:00
greatroar
3b24c15c3d
fuse: Mix inode hashes in a non-symmetric way
...
Since 0.15 (#4020 ), inodes are generated as hashes of names, xor'd with
the parent inode. That means that the inode of a/b/b is
h(a/b/b) = h(a) ^ h(b) ^ h(b) = h(a).
I.e., the grandchild has the same inode as the grandparent. GNU find
trips over this because it thinks it has encountered a loop in the
filesystem, and fails to search a/b/b. This happens more generally when
the same name occurs an even number of times.
Fix this by multiplying the parent by a large prime, so the combining
operation is not longer symmetric in its arguments. This is what the FNV
hash does, which we used prior to 0.15. The hash is now
h(a/b/b) = h(b) ^ p*(h(b) ^ p*h(a))
Note that we already ensure that h(x) is never zero.
Collisions can still occur, but they should be much less likely to occur
within a single path.
Fixes #4253 .
2023-04-14 20:50:39 +02:00
greatroar
4304e01ca2
fuse: Report fuse.Attr.Blocks correctly
...
Fixes #4239 .
2023-04-14 20:50:08 +02:00
Torben Giesselmann
6d6c04abef
forget: Simplify usage text
2023-04-14 10:05:23 -07:00
greatroar
49e32f3f8a
ui/termstatus: Optimize Truncate
...
x/text/width.LookupRune has to re-encode its argument as UTF-8,
while LookupString operates on the UTF-8 directly.
The uint casts get rid of a bounds check.
Benchmark results, with b.ResetTimer introduced first:
name old time/op new time/op delta
TruncateASCII-8 69.7ns ± 1% 55.2ns ± 1% -20.90% (p=0.000 n=20+18)
TruncateUnicode-8 350ns ± 1% 171ns ± 1% -51.05% (p=0.000 n=20+19)
2023-04-14 11:13:39 +02:00
greatroar
9412f37e50
ui/termstatus: Quote funny filenames
...
Fixes #2260 , #4191 .
2023-04-14 09:36:02 +02:00
Ian Muge
593eb710b4
added changelog
2023-04-13 22:54:36 +02:00
greatroar
97274ecabd
cmd, restic: Refactor and fix snapshot filtering
...
This turns snapshotFilterOptions from cmd into a restic.SnapshotFilter
type and makes restic.FindFilteredSnapshot and FindFilteredSnapshots
methods on that type. This fixes #4211 by ensuring that hosts and paths
are named struct fields instead of unnamed function arguments in long
lists of such.
Timestamp limits are also included in the new type. To avoid too much
pointer handling, the convention is that time zero means no limit.
That's January 1st, year 1, 00:00 UTC, which is so unlikely a date that
we can sacrifice it for simpler code.
2023-04-13 22:51:45 +02:00
Michael Eischer
74f7dd0b38
Make help for --verbose less confusing
...
The output is now
```
-v, --verbose be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)
```
instead of
```
-v, --verbose n be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)
```
2023-04-13 22:46:37 +02:00
Michael Eischer
21ad357c10
add linux/riscv64 builds
2023-04-13 22:45:35 +02:00
Heiko Schlittermann (HS12-RIPE)
087cf7e114
generate: write progress to STDOUT if this is a terminal
...
This allows usage as:
eval "$(restic generated --bash-completion /dev/stdout)"
2023-04-12 22:39:59 +02:00
Michael Eischer
e604939e72
Debug log status code if execution is interrupted
...
Currently, there is no clear indication in the debug log whether restic
exited or not.
2023-04-12 21:58:35 +02:00
Michael Eischer
37aca6bec0
Add warnings via Warnf to the debug log
2023-04-12 21:37:37 +02:00
Michael Eischer
bdcafbc11c
Merge pull request #4282 from TorstenC/patch-1
...
040_backup.rst line 228 - #4280 --exclude-caches
2023-04-11 22:34:16 +02:00
Torsten C
ba33e41068
040_backup.rst line 228 - #4280 --exclude-caches
...
Proposal for issue #4280
2023-04-11 22:33:10 +02:00
Michael Eischer
4661f45a8c
Merge pull request #4220 from Skaronator/master
...
Add jq to container image
2023-04-11 22:26:02 +02:00
Michael Eischer
2091fc0dde
Merge pull request #4107 from jooola/feature-wait-for-unlock
...
Add a global option --retry-lock
2023-04-11 22:21:01 +02:00
Michael Eischer
9238dcc81a
Merge pull request #4201 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/storage/azblob-1.0.0
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob from 0.5.1 to 1.0.0
2023-04-11 22:20:38 +02:00
Torben Giesselmann
71537da4b0
init: Add --copy-chunker-params verbose msg
2023-04-09 14:59:07 -07:00
Torben Giesselmann
ce51d2f3c0
forget: Update usage text
2023-04-09 12:59:15 -07:00
Torben Giesselmann
403b7ca2be
forget: Update documentation
2023-04-09 12:57:37 -07:00
Torben Giesselmann
309cf0586a
Add changelog entry
2023-04-09 12:05:15 -07:00
Torben Giesselmann
9f9e91eb0d
Fix comment to comply with linter
2023-04-09 11:47:10 -07:00
Michael Eischer
1f43003cc1
Merge branch 'master' into feature-wait-for-unlock
2023-04-07 20:32:21 +02:00
Michael Eischer
8ce5f29758
lock: increase test timeout tolerances to avoid test failures
2023-04-07 20:07:05 +02:00
Michael Eischer
8afc117aa3
Merge pull request #4285 from MichaelEischer/debug-lock-refresh-failures
...
lock: Improve debug logging in the test
2023-04-07 20:06:15 +02:00
Michael Eischer
cf1cc1fb72
lock: Print stacktrace if TestLockSuccessfulRefresh fails
2023-04-07 19:51:31 +02:00
Michael Eischer
64233ca0a7
lock: Improve debug logging in the test
2023-04-07 19:43:00 +02:00
jo
ea59896bd6
Add a global option --retry-lock
...
Fixes restic#719
If the option is passed, restic will wait the specified duration of time
and retry locking the repo every 10 seconds (or more often if the total
timeout is relatively small).
- Play nice with json output
- Reduce wait time in lock tests
- Rework timeout last attempt
- Reduce test wait time to 0.1s
- Use exponential back off for the retry lock
- Don't pass gopts to lockRepo functions
- Use global variable for retry sleep setup
- Exit retry lock on cancel
- Better wording for flag help
- Reorder debug statement
- Refactor tests
- Lower max sleep time to 1m
- Test that we cancel/timeout in time
- Use non blocking sleep function
- Refactor into minDuration func
Co-authored-by: Julian Brost <julian@0x4a42.net>
2023-04-07 16:24:14 +02:00
Michael Eischer
806a0cdce3
bump minimum go version on Solaris to 1.20
2023-04-07 15:40:57 +02:00
Michael Eischer
faa83db9e4
azure: Adapt code to API change
2023-04-07 15:05:55 +02:00
dependabot[bot]
9358a5fb37
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/storage/azblob](https://github.com/Azure/azure-sdk-for-go ) from 0.5.1 to 1.0.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/armcore/v0.5.1...v1.0 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-04-07 11:03:25 +00:00
Michael Eischer
71c9516b26
Merge pull request #4247 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.4.0
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.3.1 to 1.4.0
2023-04-07 13:01:04 +02:00
Michael Eischer
17ac91fba3
Merge pull request #4283 from restic/dependabot/go_modules/google.golang.org/api-0.116.0
...
build(deps): bump google.golang.org/api from 0.114.0 to 0.116.0
2023-04-07 12:58:54 +02:00
Michael Eischer
26a3c47c5c
Merge pull request #4255 from greatroar/fuse-hash
...
fuse: Mix inode hashes in a non-symmetric way
2023-04-07 12:56:57 +02:00
Michael Eischer
7896e50301
Merge pull request #4271 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.50
...
build(deps): bump github.com/minio/minio-go/v7 from 7.0.49 to 7.0.50
2023-04-07 12:47:30 +02:00
dependabot[bot]
9584cbda90
build(deps): bump google.golang.org/api from 0.114.0 to 0.116.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.114.0 to 0.116.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.114.0...v0.116.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-04-07 10:27:38 +00:00
dependabot[bot]
cacc48fc09
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.3.1 to 1.4.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.3.1...sdk/azcore/v1.4.0 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-04-07 10:27:30 +00:00
Michael Eischer
7fbaca577b
Merge pull request #4272 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.30.1
...
build(deps): bump cloud.google.com/go/storage from 1.29.0 to 1.30.1
2023-04-07 12:26:20 +02:00
Michael Eischer
1f9dd84d1e
Merge pull request #3991 from mark-herrmann/feature/restore-progress
...
restore: show progress
2023-04-07 12:24:55 +02:00
Michael Eischer
c8ff5592b8
Merge pull request #4273 from ducalex/gitignore-restic-exe
...
Added restic.exe to the .gitignore
2023-04-07 12:10:09 +02:00
Mark Herrmann
f875a8843d
restore: Add progress bar
...
Co-authored-by: Mark Herrmann <mark.herrmann@mailbox.org>
2023-04-07 12:08:23 +02:00
Alex Duchesne
0ed5c20c57
Added restic.exe to the .gitignore
2023-03-28 23:07:25 -04:00
dependabot[bot]
21edbdc3ac
build(deps): bump cloud.google.com/go/storage from 1.29.0 to 1.30.1
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.29.0 to 1.30.1.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.29.0...spanner/v1.30.1 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-26 20:14:09 +00:00
dependabot[bot]
220d937975
build(deps): bump github.com/minio/minio-go/v7 from 7.0.49 to 7.0.50
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.49 to 7.0.50.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.49...v7.0.50 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-26 20:13:34 +00:00
rawtaz
5f13bbc118
Merge pull request #4270 from MichaelEischer/monthly-dependabot
...
Switch back to monthly dependabot updates
2023-03-26 22:09:21 +02:00
Michael Eischer
9672670756
Merge pull request #4240 from greatroar/fuse-blocks
...
fuse: Report fuse.Attr.Blocks correctly
2023-03-26 21:57:53 +02:00
Michael Eischer
5ac24a9744
Switch back to monthly dependabot updates
2023-03-26 21:55:27 +02:00
Michael Eischer
99e247caa6
Merge pull request #4259 from restic/dependabot/github_actions/actions/setup-go-4
...
build(deps): bump actions/setup-go from 3 to 4
2023-03-26 21:53:47 +02:00
Michael Eischer
0c705e07db
Merge pull request #4249 from restic/dependabot/go_modules/golang.org/x/oauth2-0.6.0
...
build(deps): bump golang.org/x/oauth2 from 0.5.0 to 0.6.0
2023-03-26 10:38:53 +02:00
Michael Eischer
024d01d85b
Merge pull request #4266 from restic/linkfix
...
doc: Correct broken links
2023-03-22 22:59:54 +01:00
Leo R. Lundgren
0666fa11b8
doc: Correct broken links
2023-03-22 17:37:58 +01:00
rawtaz
caa0e89114
Merge pull request #4246 from greatroar/doc
...
doc: Fix links, focus less on implementation details
2023-03-22 14:47:31 +01:00
rawtaz
46a7072f3f
Merge pull request #4264 from GuitarBilly/patch-1
...
Update 110_talks.rst - Add changelog podcast.
2023-03-22 14:13:40 +01:00
GuitarBilly
1f12915b0c
Update 110_talks.rst
...
Update 110_talks.rst
align date to 2nd of April as agreed.
( match podcast publication date )
2023-03-22 13:54:27 +01:00
greatroar
a0885d5d69
fuse: Mix inode hashes in a non-symmetric way
...
Since 0.15 (#4020 ), inodes are generated as hashes of names, xor'd with
the parent inode. That means that the inode of a/b/b is
h(a/b/b) = h(a) ^ h(b) ^ h(b) = h(a).
I.e., the grandchild has the same inode as the grandparent. GNU find
trips over this because it thinks it has encountered a loop in the
filesystem, and fails to search a/b/b. This happens more generally when
the same name occurs an even number of times.
Fix this by multiplying the parent by a large prime, so the combining
operation is not longer symmetric in its arguments. This is what the FNV
hash does, which we used prior to 0.15. The hash is now
h(a/b/b) = h(b) ^ p*(h(b) ^ p*h(a))
Note that we already ensure that h(x) is never zero.
Collisions can still occur, but they should be much less likely to occur
within a single path.
Fixes #4253 .
2023-03-21 17:33:18 +01:00
dependabot[bot]
5dccab701a
build(deps): bump actions/setup-go from 3 to 4
...
Bumps [actions/setup-go](https://github.com/actions/setup-go ) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases )
- [Commits](https://github.com/actions/setup-go/compare/v3...v4 )
---
updated-dependencies:
- dependency-name: actions/setup-go
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 02:18:58 +00:00
Torben Giesselmann
5069c9edd9
Represent -1 as "all" in ExpirePolicy's Stringer
2023-03-15 15:07:51 -07:00
Torben Giesselmann
1a584cb16e
Refactor policy sum calculation & duration parsing
...
- Convert policy sum calculation to function and move to tests.
- Remove parseDuration(...) and use ParseDurationOrPanic(...) instead.
2023-03-14 19:29:08 -07:00
Torben Giesselmann
84ede6ad7a
forget: Prevent neg. values in --keep-within* opts
2023-03-14 19:20:03 -07:00
Torben Giesselmann
b7f03d01b8
Add helper function for Duration parsing
...
Tests in cmd_forget_test.go need the same convenience function
that was implemented in snapshot_policy_test.go.
Function parseDuration(...) was moved to testing.go and renamed to
ParseDurationOrPanic(...).
2023-03-14 19:16:24 -07:00
dependabot[bot]
eaceaca113
build(deps): bump golang.org/x/oauth2 from 0.5.0 to 0.6.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.5.0 to 0.6.0.
- [Release notes](https://github.com/golang/oauth2/releases )
- [Commits](https://github.com/golang/oauth2/compare/v0.5.0...v0.6.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-13 02:07:39 +00:00
greatroar
a7ac9a4769
doc: Fix some links, focus less on implementation details
...
The godoc for filepath.Match has the syntax, which is what is important
for writing patterns. Use pkg.go.dev instead of golang.org/pkg.
For #4245 . Not all links fixed yet.
2023-03-13 00:23:15 +01:00
Michael Eischer
23ed03a267
Merge pull request #4243 from restic/update-go
...
Run tests with Go 1.20
2023-03-12 18:15:54 +01:00
Alexander Neumann
aac8c5a7ca
Merge pull request #4244 from MichaelEischer/fix-doc-typos
...
Fix documentation typos and build with Sphinx 6
2023-03-12 15:04:26 +01:00
Michael Eischer
7c8a401d97
Merge pull request #4176 from LXGaming/fix/impostor
...
Fix scan_finished JSON MessageType
2023-03-12 12:14:40 +01:00
Alexander Neumann
d83332315c
Update golangci-lint
2023-03-12 12:05:59 +01:00
Alexander Neumann
c2703e5024
Increase timeout for golangci-lint
2023-03-12 12:02:22 +01:00
Michael Eischer
eb7dbc88b5
Merge pull request #4210 from aawsome/patch-1
...
docs: Corrections/extra information in design.rst
2023-03-12 12:00:25 +01:00
Michael Eischer
e02aadf1d2
doc: use extlink for links to issues
2023-03-12 11:55:36 +01:00
Michael Eischer
c4e6b198ae
doc: fix building with Sphinx 6
2023-03-12 11:45:26 +01:00
Michael Eischer
f47c8eebb7
doc: fix broken link in rewrite command docs
2023-03-12 11:44:56 +01:00
Alexander Neumann
996e2ac7c5
Run tests with Go 1.20
2023-03-12 11:41:22 +01:00
Michael Eischer
a67d3781a3
doc: apply review comments and improve link formatting
2023-03-12 11:37:37 +01:00
Michael Eischer
153a73ebba
Merge pull request #4242 from MichaelEischer/fix-mixed-index-in-design-docs
...
doc: Remove mixed pack files from index example
2023-03-12 11:24:21 +01:00
Michael Eischer
f9d6e3a035
Merge pull request #4236 from restic/dependabot/go_modules/golang.org/x/crypto-0.7.0
...
build(deps): bump golang.org/x/crypto from 0.5.0 to 0.7.0
2023-03-11 20:42:38 +01:00
dependabot[bot]
51656e8764
build(deps): bump golang.org/x/crypto from 0.5.0 to 0.7.0
...
Bumps [golang.org/x/crypto](https://github.com/golang/crypto ) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/crypto/releases )
- [Commits](https://github.com/golang/crypto/compare/v0.5.0...v0.7.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/crypto
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-11 19:17:59 +00:00
Michael Eischer
9e23200bff
Merge pull request #4219 from ianmuge/feat_upgrade_minio_for_new_region
...
(feat) Upgrade Minio to 7.0.49
2023-03-11 20:17:03 +01:00
Michael Eischer
b884643b40
Merge pull request #4224 from restic/dependabot/go_modules/github.com/klauspost/compress-1.16.0
...
build(deps): bump github.com/klauspost/compress from 1.15.15 to 1.16.0
2023-03-11 20:14:55 +01:00
Ian Muge
5be4845710
added changelog
2023-03-11 20:04:12 +01:00
Ian Muge
cfa3c6abc5
upgrade Minio to 7.0.49
2023-03-11 20:04:12 +01:00
Michael Eischer
f499e66032
Merge pull request #4235 from restic/dependabot/go_modules/google.golang.org/api-0.111.0
...
build(deps): bump google.golang.org/api from 0.109.0 to 0.111.0
2023-03-11 20:02:03 +01:00
Michael Eischer
00575ecffe
doc: Remove mixed pack files from index example
...
The example given for the format of an index shows a mixed pack file.
Mixing tree and data blobs has been deprecated for a long time. Thus,
change the pack to only contain "data" blobs.
2023-03-11 19:10:31 +01:00
greatroar
cb5694d136
fuse: Report fuse.Attr.Blocks correctly
...
Fixes #4239 .
2023-03-07 22:14:07 +01:00
dependabot[bot]
100b06d806
build(deps): bump google.golang.org/api from 0.109.0 to 0.111.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.109.0 to 0.111.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.109.0...v0.111.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-03-06 02:11:38 +00:00
Torben Giesselmann
667536cea4
forget: Allow neg. values in "--keep-within*" opts
2023-03-05 14:18:08 -08:00
Torben Giesselmann
ba183c44c3
forget: Add test with regular and "forever" opts
2023-03-05 13:56:16 -08:00
Torben Giesselmann
32e6a438be
forget: Add test for "keep all hourly snapshots"
2023-03-04 22:13:34 -08:00
Torben Giesselmann
b77b0749fa
forget: Treat -1 as forever for all "last n" opts
2023-03-04 19:01:37 -08:00
Torben Giesselmann
6aca7dac21
forget: Verify forget opts
2023-03-03 19:10:11 -08:00
Torben Giesselmann
8161605f1b
snapshot_group: Fix typo
2023-03-03 19:07:57 -08:00
dependabot[bot]
b78607c9d8
build(deps): bump github.com/klauspost/compress from 1.15.15 to 1.16.0
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.15.15 to 1.16.0.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.15.15...v1.16.0 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-27 02:52:10 +00:00
Niklas Wagner
c1101ede19
Add jq to container image
2023-02-21 14:01:43 +01:00
Michael Eischer
f646406822
Merge pull request #4182 from BenWiederhake/dev-with-cache-description
...
check: Document behavior of --with-cache a bit better
2023-02-19 19:41:38 +01:00
Ben Wiederhake
9888443f5c
check: Document behavior of --with-cache a bit better
...
Also see https://forum.restic.net/t/how-to-speed-up-tiny-incremental-checks/5905
2023-02-19 17:45:47 +01:00
Michael Eischer
cf6dfd6d36
Merge pull request #4212 from greatroar/snapshotfilter
...
cmd, restic: Refactor and fix snapshot filtering
2023-02-19 15:20:18 +01:00
greatroar
a7786c67f1
cmd, restic: Refactor and fix snapshot filtering
...
This turns snapshotFilterOptions from cmd into a restic.SnapshotFilter
type and makes restic.FindFilteredSnapshot and FindFilteredSnapshots
methods on that type. This fixes #4211 by ensuring that hosts and paths
are named struct fields instead of unnamed function arguments in long
lists of such.
Timestamp limits are also included in the new type. To avoid too much
pointer handling, the convention is that time zero means no limit.
That's January 1st, year 1, 00:00 UTC, which is so unlikely a date that
we can sacrifice it for simpler code.
2023-02-19 15:04:25 +01:00
Michael Eischer
15b7d9c80b
Merge pull request #4213 from restic/dependabot/go_modules/golang.org/x/net-0.7.0
...
build(deps): bump golang.org/x/net from 0.6.0 to 0.7.0
2023-02-19 12:42:01 +01:00
dependabot[bot]
ee4128281e
build(deps): bump golang.org/x/net from 0.6.0 to 0.7.0
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.6.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases )
- [Commits](https://github.com/golang/net/compare/v0.6.0...v0.7.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-19 11:32:55 +00:00
Michael Eischer
3c8aefa0cb
Merge pull request #4205 from restic/dependabot/go_modules/golang.org/x/oauth2-0.5.0
...
build(deps): bump golang.org/x/oauth2 from 0.4.0 to 0.5.0
2023-02-19 12:31:43 +01:00
aawsome
e2df73b0ac
Update doc/design.rst
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2023-02-17 10:07:53 +01:00
aawsome
f4329a20f6
docs: Corrections/extra information in design.rst
2023-02-16 22:41:30 +01:00
dependabot[bot]
11ebc0c5db
build(deps): bump golang.org/x/oauth2 from 0.4.0 to 0.5.0
...
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2 ) from 0.4.0 to 0.5.0.
- [Release notes](https://github.com/golang/oauth2/releases )
- [Commits](https://github.com/golang/oauth2/compare/v0.4.0...v0.5.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-13 02:54:09 +00:00
Michael Eischer
f137be42fe
Merge pull request #4081 from MichaelEischer/backup-group-by
...
Add support for `--group-by` to the backup command
2023-02-11 11:38:13 +01:00
Michael Eischer
0ce182f044
document backup --group-by
2023-02-11 00:55:11 +01:00
Alex Thomson
1b50faf03e
Add changelog
2023-02-10 23:45:20 +01:00
Alex Thomson
4cbbf5d952
Fix scan_finished JSON MessageType
...
Corrected the scan_finished JSON MessageType from status to verbose_status
2023-02-10 23:45:20 +01:00
Michael Eischer
2885db7902
backup: add group-by option
2023-02-10 23:18:14 +01:00
Michael Eischer
acb40d2b94
Refactor group-by to parse options into a struct
2023-02-10 23:18:14 +01:00
Michael Eischer
fa73b50b45
Merge pull request #4198 from MichaelEischer/better-verbose-description
...
Make help for --verbose less confusing
2023-02-09 22:28:18 +01:00
Michael Eischer
2d700c3887
Make help for --verbose less confusing
...
The output is now
```
-v, --verbose be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)
```
instead of
```
-v, --verbose n be verbose (specify multiple times or a level using --verbose=n, max level/times is 2)
```
2023-02-06 22:11:21 +01:00
Michael Eischer
91251f2d57
Merge pull request #4196 from restic/dependabot/github_actions/docker/build-push-action-4
...
build(deps): bump docker/build-push-action from 3 to 4
2023-02-06 21:54:06 +01:00
Michael Eischer
3df4ec7c61
Merge pull request #4195 from restic/dependabot/go_modules/github.com/Azure/azure-sdk-for-go/sdk/azcore-1.3.1
...
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore from 1.3.0 to 1.3.1
2023-02-06 21:20:33 +01:00
Michael Eischer
b1d3a1a5e3
Merge pull request #4194 from restic/dependabot/go_modules/google.golang.org/api-0.109.0
...
build(deps): bump google.golang.org/api from 0.108.0 to 0.109.0
2023-02-06 21:17:57 +01:00
dependabot[bot]
4f31c2699d
build(deps): bump docker/build-push-action from 3 to 4
...
Bumps [docker/build-push-action](https://github.com/docker/build-push-action ) from 3 to 4.
- [Release notes](https://github.com/docker/build-push-action/releases )
- [Commits](https://github.com/docker/build-push-action/compare/v3...v4 )
---
updated-dependencies:
- dependency-name: docker/build-push-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-06 02:13:17 +00:00
dependabot[bot]
96b1ff5e38
build(deps): bump github.com/Azure/azure-sdk-for-go/sdk/azcore
...
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azcore](https://github.com/Azure/azure-sdk-for-go ) from 1.3.0 to 1.3.1.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases )
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md )
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/azcore/v1.3.0...sdk/azcore/v1.3.1 )
---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azcore
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-06 02:07:34 +00:00
dependabot[bot]
6b5ffce9dc
build(deps): bump google.golang.org/api from 0.108.0 to 0.109.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.108.0 to 0.109.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.108.0...v0.109.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-02-06 02:06:57 +00:00
Michael Eischer
590eb9efd7
Merge pull request #4180 from MichaelEischer/add-riscv
...
Add linux/riscv64 builds
2023-01-30 22:57:39 +01:00
Michael Eischer
55c4ca66f7
add linux/riscv64 builds
2023-01-30 22:24:12 +01:00
Michael Eischer
56ad761b19
Merge pull request #4177 from greatroar/loadunpacked
...
repository: Reuse buffers in Repository.LoadUnpacked
2023-01-30 22:12:13 +01:00
greatroar
d129baba7a
repository: Reuse buffers in Repository.LoadUnpacked
...
This method had a buffer argument, but that was nil at all call sites.
That's removed, and instead LoadUnpacked now reuses whatever it
allocates inside its retry loop.
2023-01-30 22:01:01 +01:00
Alexander Neumann
febb32b5b4
Set development version for 0.15.1
2023-01-30 20:43:52 +01:00
Alexander Neumann
7d4b7ad9cb
Add version for 0.15.1
2023-01-30 20:43:47 +01:00
Alexander Neumann
a883bb6596
Generate CHANGELOG.md for 0.15.1
2023-01-30 20:43:15 +01:00
Alexander Neumann
91acef90b2
Prepare changelog for 0.15.1
2023-01-30 20:43:15 +01:00
Michael Eischer
b2b7727b31
Merge pull request #4178 from rawtaz/changelogs
...
Polish changelogs
2023-01-27 22:46:46 +01:00
Leo R. Lundgren
0e4c9a5421
Polish changelogs
2023-01-27 22:27:57 +01:00
Michael Eischer
49fa8fe6dd
Merge pull request #4175 from fergus-dall/deadlock-fix
...
Fix deadlock in Lock.Stale
2023-01-26 22:45:13 +01:00
Michael Eischer
12f167ee79
Merge pull request #4167 from aneesh-n/progress-percent-eta-fix
...
ui/backup: Fix percent and eta in backup progress
2023-01-26 22:42:10 +01:00
Michael Eischer
bb018fbc3e
Merge pull request #4163 from MichaelEischer/fix-windows-self-upgrade
...
self-upgrade: Fix handling of `--output` on windows
2023-01-26 22:40:36 +01:00
Michael Eischer
3b24e0ac55
Merge pull request #4154 from MichaelEischer/cleanup-prune-tests
...
deduplicate prune tests a bit
2023-01-26 22:39:15 +01:00
Fergus Dall
04da31af2b
Fix deadlock in Lock.Stale
...
With debug logging enabled this method would take a lock and then
format the lock as a string. Since PR #4022 landed the string
formatting method has also taken the lock, so this deadlocks.
Instead just record the lock ID, as is done elsewhere.
2023-01-27 03:21:37 +11:00
Alexander Neumann
65923e9c26
Merge pull request #4152 from MichaelEischer/invalid-locks
...
Improve handling of invalid locks
2023-01-25 08:19:53 +01:00
Michael Eischer
b903081804
Merge pull request #4169 from restic/dependabot/go_modules/github.com/klauspost/compress-1.15.15
...
build(deps): bump github.com/klauspost/compress from 1.15.14 to 1.15.15
2023-01-23 22:35:52 +01:00
Michael Eischer
beb1e872cc
Merge pull request #4170 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.29.0
...
build(deps): bump cloud.google.com/go/storage from 1.28.1 to 1.29.0
2023-01-23 22:25:09 +01:00
Michael Eischer
db350c0430
Merge pull request #4171 from restic/dependabot/go_modules/google.golang.org/api-0.108.0
...
build(deps): bump google.golang.org/api from 0.107.0 to 0.108.0
2023-01-23 22:23:51 +01:00
dependabot[bot]
716a5dd20d
build(deps): bump google.golang.org/api from 0.107.0 to 0.108.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.107.0 to 0.108.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.107.0...v0.108.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-23 02:23:51 +00:00
dependabot[bot]
dbd07ade98
build(deps): bump cloud.google.com/go/storage from 1.28.1 to 1.29.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.28.1 to 1.29.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/storage/v1.28.1...spanner/v1.29.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-23 02:23:33 +00:00
dependabot[bot]
7adf1e5d37
build(deps): bump github.com/klauspost/compress from 1.15.14 to 1.15.15
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.15.14 to 1.15.15.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.15.14...v1.15.15 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-23 02:23:15 +00:00
Michael Eischer
8f94eb5420
add changelog for less strict lock handling
2023-01-22 15:54:07 +01:00
Michael Eischer
8aaba83719
add changelog for self-upgrade bug on windows
2023-01-22 15:39:42 +01:00
Michael Eischer
e16a6d4c50
self-update: add basic test for extractToFile
2023-01-22 15:39:42 +01:00
Michael Eischer
34e67e3510
self-update: Fix handling of `--output` on windows
...
The code always assumed that the upgrade happens in place. Thus writing
the upgrade to a separate file fails, when trying to remove the file
stored at that location.
2023-01-22 15:39:42 +01:00
Michael Eischer
c527c05590
add changelog for missing backup ETA
2023-01-22 15:32:55 +01:00
Aneesh Nireshwalia
ed23edeb62
ui/backup: Fix percent and eta in backup progress
...
Added missing call to scanFinished=true.
This was causing the percent and eta to never get
printed for backup progress even after the scan was finished.
2023-01-21 13:25:08 -07:00
Michael Eischer
0f398b82e3
Merge pull request #4145 from greatroar/index-encode
...
index: Optimize generatePackList
2023-01-21 01:01:12 +01:00
greatroar
99755c634b
index: Optimize generatePackList
...
name old time/op new time/op delta
EncodeIndex/100-8 1.56ms ± 2% 1.48ms ± 3% -5.37% (p=0.000 n=10+10)
EncodeIndex/1000-8 14.5ms ± 2% 13.1ms ± 2% -9.49% (p=0.000 n=9+10)
EncodeIndex/10000-8 120ms ± 2% 116ms ± 2% -3.58% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
EncodeIndex/100-8 306kB ± 1% 275kB ± 1% -10.28% (p=0.000 n=10+10)
EncodeIndex/1000-8 3.69MB ±11% 2.88MB ± 5% -22.07% (p=0.000 n=10+9)
EncodeIndex/10000-8 35.9MB ±11% 31.9MB ±10% -11.13% (p=0.005 n=10+10)
name old allocs/op new allocs/op delta
EncodeIndex/100-8 3.39k ± 0% 2.39k ± 0% -29.61% (p=0.000 n=10+10)
EncodeIndex/1000-8 32.6k ± 0% 22.9k ± 0% -29.63% (p=0.000 n=10+9)
EncodeIndex/10000-8 326k ± 0% 229k ± 0% -29.71% (p=0.000 n=10+10)
The bulk of the allocation rate improvement comes from just removing the
debug.Log calls: every one of those copied a restic.ID to the heap.
2023-01-14 20:41:07 +01:00
Michael Eischer
f5f13f6648
Merge pull request #4153 from MichaelEischer/fix-quiet-prune
...
prune: Don't show "packs processed" for quiet runs
2023-01-14 20:11:34 +01:00
Michael Eischer
00216d54a1
deduplicate prune tests a bit
2023-01-14 19:05:45 +01:00
Michael Eischer
1f3f68b2c0
prune: Don't show "packs processed" for quiet runs
2023-01-14 18:48:32 +01:00
Michael Eischer
57acc769b4
lock: Ignore empty lock files
...
These may be left behind by backends which do not guarantee atomic
uploads.
2023-01-14 18:15:46 +01:00
Michael Eischer
20ad14e362
lock: add help message how to recover from invalid locks
2023-01-14 18:04:22 +01:00
Michael Eischer
c995b5be52
lock: cleanup error message
...
The error message is now `Fatal: unable to create lock in backend:
[...]` instead of `unable to create lock in backend: Fatal: [...]`.
2023-01-14 17:57:02 +01:00
Michael Eischer
1adf28a2b5
repository: properly return invalid data error in LoadUnpacked
...
The retry backend does not return the original error, if its execution
is interrupted by canceling the context. Thus, we have to manually
ensure that the invalid data error gets returned.
Additionally, use the retry backend for some of the repository tests, as
this is the configuration which will be used by restic.
2023-01-14 17:57:02 +01:00
Michael Eischer
6d9675c323
repository: cleanup error message on invalid data
...
The retry printed the filename twice:
```
Load(<lock/04804cba82>, 0, 0) returned error, retrying after 720.254544ms: load(<lock/04804cba82>): invalid data returned
```
now the warning has changed to
```
Load(<lock/04804cba82>, 0, 0) returned error, retrying after 720.254544ms: invalid data returned
```
2023-01-14 17:57:02 +01:00
Michael Eischer
551b31ce3c
Merge pull request #4146 from MichaelEischer/update-blazer
...
Update blazer to fix `b2_download_file_by_name: 404`
2023-01-14 15:23:32 +01:00
Michael Eischer
ec99507e4c
update blazer
...
This removes a stray warning, that was printed when checking the status
of non-existent files.
2023-01-14 14:56:25 +01:00
Michael Eischer
5f97f534b1
Merge pull request #4150 from restic/dependabot/go_modules/google.golang.org/api-0.107.0
...
build(deps): bump google.golang.org/api from 0.106.0 to 0.107.0
2023-01-14 14:55:15 +01:00
Michael Eischer
ed11bbd0e2
Merge pull request #4151 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.47
...
build(deps): bump github.com/minio/minio-go/v7 from 7.0.46 to 7.0.47
2023-01-14 14:50:01 +01:00
dependabot[bot]
5bb9cb056d
build(deps): bump github.com/minio/minio-go/v7 from 7.0.46 to 7.0.47
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.46 to 7.0.47.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.46...v7.0.47 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-14 13:38:58 +00:00
dependabot[bot]
cd9bd22563
build(deps): bump google.golang.org/api from 0.106.0 to 0.107.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.106.0 to 0.107.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.106.0...v0.107.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2023-01-14 13:38:42 +00:00
Michael Eischer
ecc826ef7d
Merge pull request #4149 from MichaelEischer/more-frequent-dependabot
...
CI: let dependabot check for updates weekly
2023-01-14 14:37:30 +01:00
Michael Eischer
fb43cbab49
Merge pull request #4129 from greatroar/cleanup
...
cache: Replace readCloser+LimitedReader by backend.LimitedReadCloser
2023-01-14 12:17:25 +01:00
Michael Eischer
41d31b1e27
Merge pull request #4116 from MichaelEischer/extract-progress-updater
...
ui/progress: Extract progress updater
2023-01-14 12:07:26 +01:00
Michael Eischer
f6ea5c5865
CI: let dependabot check for updates weekly
2023-01-14 12:02:42 +01:00
Michael Eischer
4a7a6b06af
ui/backup: Use progress.Updater for progress updates
2023-01-14 01:20:43 +01:00
Michael Eischer
e499bbe3ae
progress: extract progress updating into Updater struct
...
This allows reusing the code to create periodic progress updates.
2023-01-14 01:13:08 +01:00
Michael Eischer
52682b1c7b
Merge pull request #4111 from MichaelEischer/extract-stdio-wrapper
...
backup: extract StdioWrapper from ProgressPrinters
2023-01-14 01:12:23 +01:00
Michael Eischer
c15b4bceae
backup: extract StdioWrapper from ProgressPrinters
...
The StdioWrapper is not used at all by the ProgressPrinters. It is
called a bit earlier than previously. However, as the password prompt
directly accessed stdin/stdout this doesn't cause problems.
2023-01-14 00:58:13 +01:00
Alexander Neumann
74348be3fa
Set development version for 0.15.0
2023-01-12 20:51:23 +01:00
Alexander Neumann
37d0e1fe58
Add version for 0.15.0
2023-01-12 20:51:19 +01:00
Alexander Neumann
da196aa43e
Update manpages and auto-completion
2023-01-12 20:51:19 +01:00
Alexander Neumann
099774c2aa
Generate CHANGELOG.md for 0.15.0
2023-01-12 20:50:45 +01:00
Alexander Neumann
cd2f53e3f9
Prepare changelog for 0.15.0
2023-01-12 20:50:44 +01:00
Alexander Neumann
0c5a55d1bd
Merge pull request #4142 from restic/update-deps
...
Upgrade dependencies
2023-01-12 20:23:25 +01:00
Michael Eischer
9ddca65f6d
update dependencies
2023-01-11 23:22:10 +01:00
Michael Eischer
06fee601bc
Merge pull request #4141 from restic/doc-no-scan-correction
...
doc: Correct heading level for --no-scan
2023-01-11 22:01:05 +01:00
Leo R. Lundgren
1cb920cc57
doc: Correct heading level for --no-scan
2023-01-11 21:47:38 +01:00
Michael Eischer
8f53ffb921
Merge pull request #4140 from restic/doc-manual
...
doc: Update manual page with --no-scan
2023-01-11 21:42:42 +01:00
Michael Eischer
351cbb4f94
Merge pull request #4139 from restic/doc-no-scan
...
doc: Move and update documentation for --no-scan
2023-01-11 21:41:20 +01:00
Leo R. Lundgren
b8b5508d15
doc: Update manual page with --no-scan
2023-01-11 02:27:43 +01:00
Leo R. Lundgren
c5542ddcd2
doc: Move and update documentation for --no-scan
2023-01-11 00:09:24 +01:00
rawtaz
dffb8e0c14
Merge pull request #4138 from MichaelEischer/doc-sparse
...
doc: add description for restore --sparse
2023-01-10 23:39:51 +01:00
Michael Eischer
b151fa498a
doc: add description for restore --sparse
2023-01-10 23:27:42 +01:00
Michael Eischer
c354b55e62
Merge pull request #4136 from restic/doc-small-files
...
doc: Clarify text about tuning backups for small files
2023-01-08 22:00:09 +01:00
Michael Eischer
375953a001
Merge pull request #4088 from MichaelEischer/doc-cifs-backup-source
...
doc: reading from CIFS can be a problem on linux
2023-01-08 21:56:48 +01:00
Leo R. Lundgren
6306797238
doc: Clarify text about tuning backups for small files
2023-01-08 21:49:55 +01:00
Michael Eischer
ef9164fcbb
doc: reading from CIFS can be a problem on linux
2023-01-08 21:47:34 +01:00
rawtaz
e2bcfd68dd
Merge pull request #4135 from restic/changelogs
...
Polish changelogs
2023-01-08 20:04:24 +01:00
Leo R. Lundgren
33fb351386
Polish changelogs
2023-01-08 19:48:51 +01:00
Michael Eischer
c9840da4f8
Merge pull request #4134 from MichaelEischer/fix-verbose-help-text
...
Correct maximum verbosity level in help message
2023-01-07 22:24:04 +01:00
Michael Eischer
732184a849
Correct maximum verbosity level in help message
...
The maximum for `--verbose=n` is n=2. Internally it is translated into a
scale from 0 to 3. However, the default (without verbose) is 1, thus the
verbosity level can only be increased two times.
2023-01-07 22:02:13 +01:00
Michael Eischer
24178c97e9
Merge pull request #4117 from MichaelEischer/prune-dry-run-help
...
prune: make it clearer when prune is used in dry-run mode
2023-01-04 23:18:53 +01:00
Michael Eischer
7a36306901
forget: Clarify log message for `--dry-run --prune`
2023-01-04 00:44:46 +01:00
Michael Eischer
b404ad4eaa
prune: make it clearer when prune is used in dry-run mode
2023-01-04 00:44:46 +01:00
Michael Eischer
e02a10c58a
Merge pull request #4109 from MichaelEischer/fix-prune-uncompressed-accounting
...
prune: Fix calculation of remaining uncompressed data
2023-01-03 23:28:10 +01:00
Michael Eischer
81dc8c8d13
prune: Fix calculation of remaining uncompressed data
...
Only the repacking of *un*compressed packs reduces the amount of
uncompressed data. Previously the counter even overflowed for fully
compressed repositories.
2023-01-03 22:34:36 +01:00
greatroar
72922a79ed
cache: Replace readCloser+LimitedReader by backend.LimitedReadCloser
2023-01-03 19:03:36 +01:00
Michael Eischer
89a8006578
Merge pull request #4104 from philaris/fix_max_uint32_uid_gid_to_zero
...
in tar dump, convert uid, gid of value -1 to zero
2023-01-02 22:28:28 +01:00
Panagiotis Cheilaris
3b516d4b70
convert uid/gid -1 to 0 only in 32-bit tar dump
...
Only for a 32-bit build of restic, convert a uid or gid value of -1 to 0.
2022-12-30 18:12:12 +01:00
Michael Eischer
0de3b24756
Merge pull request #4110 from MichaelEischer/remove-exitf
...
Remove Exitf function
2022-12-29 12:07:51 +01:00
Michael Eischer
0fbff39ae8
Merge pull request #4108 from MichaelEischer/cleanup-check-output
...
Cleanup check output
2022-12-29 11:59:18 +01:00
Michael Eischer
68b1f30733
Remove Exitf function
...
Commands should use the normal shutdown path. In addition, the Exitf
function was only used by `dump` and `restore` but not any other command
which introduces the risk of inconsistent behavior.
2022-12-28 21:42:38 +01:00
Panagiotis Cheilaris
a86a56cf3b
fix lint issue with function name 'tarId'
...
See https://github.com/golang/lint/issues/89 and
https://github.com/golang/lint/issues/124
2022-12-28 18:46:58 +01:00
Panagiotis Cheilaris
050ed616ae
be more explicit with uid or gid of value -1
2022-12-28 18:44:36 +01:00
Michael Eischer
8430399fce
check: Partially fix garbled output
...
When reporting an error for a tree, the output message can overlap with
the progress bar output, e.g. `error for tree e91ef6fb:napshots`.
The fix only applies for this specific message and does not work on
Windows.
2022-12-28 17:47:27 +01:00
Michael Eischer
aea96b7d86
check: Slightly improve help message
...
If a repository has both pack/index related warnings and errors, then
the help message is quite misleading. Reword it slightly to be more
clear.
2022-12-28 17:46:06 +01:00
Michael Eischer
bcae28afb4
Merge pull request #4100 from klemensn/tag-self-update
...
Reinstate selfupdate tag to make builds without self-update work
2022-12-28 15:49:11 +01:00
Panagiotis Cheilaris
10fa5cde0a
in tar dump, convert uid, gid of value -1 to zero
2022-12-27 16:36:04 +01:00
Klemens Nanni
61e7386384
Bugfix: Make distribution package builds without self-update work
2022-12-26 21:52:24 +04:00
Klemens Nanni
94f6e7d4a6
Reinstate selfupdate tag to make builds without self-update work
...
Revert what seems to be a typo introduced as part of the fix for #2041
in 2018 7d0f2eaf24
.
`xbuild` does not look like a go build/tag keyword to me, I failed to
find documentation for it and using `go install -tags '!selfupdate' ...`
has no effect, i.e. self-update code is still compiled.
`+build` however works; updating the OpenBSD port/binary package
security/restic to apply this PR works as expected:
```
$ restic help | grep self
$ restic self-update
unknown command "self-update" for "restic"
```
(Using `go:build` now as per restic's style and gofmt.)
Previously, using `restic-0.14.0p1` on OpenBSD/amd64 7.2-current would
check for a newer version and probably attempt replacing the system wide
root-owned executable (on a read-only filesystem) as unprivileged user:
```
$ restic version
restic 0.14.0 compiled with go1.19.2 on openbsd/amd64
$ restic help | grep self
self-update Update the restic binary
$ restic self-update
writing restic to /usr/local/bin/restic
find latest release of restic at GitHub
restic is up to date
```
(It never tried to actually write besaid path; doing so would fail, so
the current message can be considered misleading.)
2022-12-26 21:46:22 +04:00
Michael Eischer
90fb6f70b4
Merge pull request #4089 from greatroar/errors
...
Clean up error handling further
2022-12-24 10:41:56 +01:00
Michael Eischer
29b8500254
Merge pull request #4090 from restic/upgrade-dependencies
...
Upgrade dependencies
2022-12-23 22:34:39 +01:00
Michael Eischer
705cabb304
Merge pull request #3981 from MichaelEischer/prune-uncompressed-stats
...
prune: report how much data must be repacked to compressed the repo
2022-12-23 22:34:04 +01:00
Michael Eischer
a6f3ae5790
Merge pull request #4094 from googol42/master
...
remove duplicated init
2022-12-23 22:33:01 +01:00
Andreas Dominik Preikschat
ea37240597
remove duplicated init
...
the documentation contained the `init` command twice
2022-12-20 17:24:56 +01:00
Michael Eischer
bd2f6aaac3
azure: downgrade azblob dependency due to build breakages on Solaris
2022-12-17 23:35:07 +01:00
Michael Eischer
583372956b
Upgrade dependencies
...
Nothing special has changed.
2022-12-17 15:23:11 +01:00
greatroar
1678392a6d
checker: Make ErrLegacyLayout a value, not a type
2022-12-17 09:41:07 +01:00
greatroar
d9002f050e
backend: Don't Wrap errors from url.Parse
...
The messages from url.Error.Error already start with the word "parse".
2022-12-17 09:41:07 +01:00
greatroar
b150dd0235
all: Replace some errors.Wrap calls by errors.WithStack
...
Mostly changed the ones that repeat the name of a system call, which is
already contained in os.PathError.Op. internal/fs.Reader had to be
changed to actually return such errors.
2022-12-17 09:41:07 +01:00
Michael Eischer
cccc17e4e9
Merge pull request #4086 from blackpiglet/modify_access_denied_code
...
Fix: change error code in function isAccessDenied to AccessDenied
2022-12-16 21:55:52 +01:00
Michael Eischer
2723159ed4
Merge pull request #3931 from kjetilho/feature/optional_scanner
...
add --no-scan to backup command
2022-12-16 21:42:22 +01:00
Xun Jiang/Bruce Jiang
ecc62c8be2
Update changelog/unreleased/issue-4085
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-12-16 21:41:16 +01:00
Xun Jiang
cc5325d22b
Fix: change error code in function isAccessDenied to AccessDenied
...
Signed-off-by: Xun Jiang <blackpiglet@gmail.com>
2022-12-16 21:41:16 +01:00
Michael Eischer
da0e45cf40
Merge pull request #4083 from greatroar/cleanup
...
repository: Remove empty cleanup functions in tests
2022-12-16 21:39:30 +01:00
Kjetil Torgrim Homme
14aa6f2a00
add --disable-scanner to backup command
...
The scanner process has only cosmetic effect for the progress printer,
and can be disabled without impacting functionality when the user does
not need an estimate of completion.
In many cases the scanner process can provide beneficial priming of
the file system cache, so as general advice it should not be disabled.
However, tests have shown that backup of NFS and fuse based filesystems,
where stat(2) is relatively expensive, can be significantly faster
without the scanner.
2022-12-16 21:29:59 +01:00
Michael Eischer
7bdb985dde
Merge pull request #4079 from MichaelEischer/rewrite-set-original
...
rewrite: Always set the Original field in a rewritten snapshot
2022-12-13 22:56:20 +01:00
Michael Eischer
1bfe98bdc0
Merge pull request #2398 from DanielG/b2-hide-file
...
b2: Fallback to b2_hide_file when delete returns unauthorized
2022-12-13 22:52:23 +01:00
Michael Eischer
1c071a462e
Merge pull request #4084 from ekarlso/azure-stat-fix
...
fix: Make create not error out when ContainerNotFound
2022-12-13 22:49:05 +01:00
Michael Eischer
25d22d5241
Merge pull request #4082 from MichaelEischer/unbuffered-logger-for-testing
...
Don't buffer the golang `log` package output when running tests
2022-12-13 22:45:50 +01:00
Endre Karlson
7dd33c0ecc
azure: Make create not error out when ContainerNotFound
2022-12-11 22:57:23 +01:00
greatroar
c0b5ec55ab
repository: Remove empty cleanup functions in tests
...
TestRepository and its variants always returned no-op cleanup functions.
If they ever do need to do cleanup, using testing.T.Cleanup is easier
than passing these functions around.
2022-12-11 11:06:25 +01:00
Michael Eischer
2e3d4640be
Don't buffer the golang log output when running tests
2022-12-10 16:08:27 +01:00
Michael Eischer
38b2e9b42c
rewrite: Always set the Original field in a rewritten snapshot
...
The Original field is meant to remember the original snapshot id if e.g.
changing its tags. It was only set by the `rewrite` command if it was
not set previously. However, a rewritten snapshot is potentially rather
different from the original snapshot. Thus just always set the Original
field. This also makes it easier to later on detect and potentially
remove the original snapshots.
2022-12-10 12:47:00 +01:00
Michael Eischer
049a105ba5
Merge pull request #4077 from greatroar/cleanup
...
test: Use testing.T.Cleanup to remove tempdirs
2022-12-09 22:17:46 +01:00
Michael Eischer
4b98b5562d
Merge pull request #4075 from greatroar/sftp-enospc
...
sftp: Fix ENOSPC check
2022-12-09 22:00:13 +01:00
greatroar
f90bf84ba7
test: Use testing.T.Cleanup to remove tempdirs
2022-12-09 14:23:55 +01:00
greatroar
83d23b3ae8
Changelog for ENOSPC handling bug
2022-12-09 08:50:30 +01:00
Michael Eischer
eae7366563
Merge pull request #4028 from ekarlso/use-az-blob-sdk
...
Switch to azblob sdk
2022-12-07 21:58:03 +01:00
Endre Karlson
25648e2501
azure: Switch to azblob sdk
2022-12-07 21:46:07 +01:00
greatroar
62520bb7b4
sftp: Fix ENOSPC check
...
We now check for space that is not reserved for the root user on the
remote, and the check is no longer in a defer block because it wouldn't
fire. Some change in the surrounding code may have led the deferred
function to capture the wrong err variable.
Fixes #3336 .
2022-12-07 21:06:46 +01:00
rawtaz
4ba31df08f
Merge pull request #4074 from greatroar/lobaro-docker
...
doc: Remove ref to Lobaro's Docker image
2022-12-04 18:00:08 +01:00
greatroar
5efcbe143c
doc: Remove ref to Lobaro's Docker image
...
It hasn't been updated for a while and has restic 0.12.0. Fixes #4002 .
2022-12-04 16:20:42 +01:00
Michael Eischer
0df585dd99
Merge pull request #4066 from sedlund/fix#4033
...
fix#4033 cmd: copy no longer lists skipped existing snapshots by default
2022-12-03 19:22:11 +01:00
Michael Eischer
223da7344e
Merge pull request #4070 from restic/fix-cloud-tests
...
Fix cloud tests
2022-12-03 19:21:25 +01:00
Michael Eischer
2b67862420
backend/test: check that IsNotExist actually works
2022-12-03 18:56:55 +01:00
Michael Eischer
2f934f5803
gs: check against the correct error in IsNotExist
2022-12-03 18:49:54 +01:00
Michael Eischer
04d101fa94
gs/s3: remove useless os.IsNotExist check
2022-12-03 18:49:54 +01:00
Michael Eischer
579cd6dc64
azure: fix totally broken IsNotExist
2022-12-03 18:49:54 +01:00
Michael Eischer
3ebdadc58f
Merge pull request #4069 from greatroar/cleanup
...
cache, prune, restic: Cleanup
2022-12-03 17:50:52 +01:00
Michael Eischer
bc8b2455b9
Merge pull request #4064 from MichaelEischer/flaky-abort-early-on-error
...
archiver: Fix flaky TestArchiverAbortEarlyOnError
2022-12-03 17:43:44 +01:00
Michael Eischer
60c6a09324
Merge pull request #4065 from MichaelEischer/flaky-rclone-failed-start
...
rclone: treat "file already closed" as command startup error
2022-12-03 17:42:56 +01:00
Michael Eischer
8bf6b2b80d
Merge pull request #4067 from MichaelEischer/remove-backend-test-method
...
Remove `Test()` method from Backend
2022-12-03 17:40:55 +01:00
Michael Eischer
78ea69082a
Merge pull request #4068 from MichaelEischer/debug-lock-refresh-test
...
Add more debug logging to `TestLockSuccessfulRefresh`
2022-12-03 17:38:21 +01:00
Scott Edlund
cbe73ace3f
Update changelog/unreleased/issue-4033
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2022-12-03 20:07:37 +08:00
greatroar
63bed34608
restic: Clean up restic.IDs type
...
IDs.Less can be rewritten as
string(list[i][:]) < string(list[j][:])
Note that this does not copy the ID's.
The Uniq method was no longer used.
The String method has been reimplemented without first copying into a
separate slice of a custom type.
2022-12-03 12:38:20 +01:00
greatroar
0c749dd358
prune: Pass fewer options around
2022-12-03 12:14:04 +01:00
greatroar
d45a2475e1
cache: Rewrite unnecessary if-else
2022-12-03 12:13:54 +01:00
Michael Eischer
6b5d6b9f2c
Add more debug logging to TestLockSuccessfulRefresh
...
The test fails from time to time. Add some more logging to hopefully get
an idea where things go wrong.
2022-12-03 12:05:38 +01:00
Michael Eischer
648edeca40
retry: Do not retry Stat() if file does not exist
...
In non test/debug code, Stat() is used exclusively to check whether a
file exists. Thus, do not retry if a file is reported as not existing.
2022-12-03 11:42:48 +01:00
Michael Eischer
40ac678252
backend: remove Test method
...
The Test method was only used in exactly one place, namely when trying
to create a new repository it was used to check whether a config file
already exists.
Use a combination of Stat() and IsNotExist() instead.
2022-12-03 11:28:10 +01:00
sedlund
06ee0339aa
fix#4033 cmd: copy no longer lists skipped existing snapshots by default
2022-12-03 09:55:39 +08:00
Michael Eischer
57d8eedb88
Merge pull request #4020 from greatroar/fuse-inode
...
fuse: Better inode generation
2022-12-02 22:28:15 +01:00
Michael Eischer
ca1803cacb
Merge pull request #4063 from MichaelEischer/replace-ioutil-usage
...
Replace ioutil usage
2022-12-02 21:49:40 +01:00
Michael Eischer
0af89a5738
Merge pull request #3132 from metalsp0rk/init-json
...
Init command JSON output
2022-12-02 21:49:22 +01:00
Michael Eischer
364a396fd6
init: use standard name `message_type` to distinguish JSON messages
2022-12-02 21:33:03 +01:00
Michael Eischer
9a9f559806
init: cleanup json print code
2022-12-02 21:33:03 +01:00
Kyle Brennan
933c9af328
create changelog entry for issue-3124 and pull-3132
2022-12-02 21:32:30 +01:00
Kyle Brennan
a6ae79b39e
support json output for init command
2022-12-02 21:32:30 +01:00
Michael Eischer
f3d964a8c1
rclone: treat "file already closed" as command startup error
...
Since #3940 the rclone backend returns the commands exit code if it
fails to start. The list of expected errors was missing the "file
already closed"-error which can occur if the http test request first
learns about the closed pipe to rclone before noticing the canceled
context.
Go internally makes sure that a file descriptor is unusable once it was
closed, thus this cannot have unintended side effects (like accidentally
reading from the wrong file due to a reused file descriptor).
2022-12-02 20:46:02 +01:00
Michael Eischer
a9972dbe7d
archiver: Fix flaky TestArchiverAbortEarlyOnError
...
Saving the blobs of a file by now happens asynchronously to the
processing in the FileSaver. Thus we have to account for the blobs
queued for saving.
2022-12-02 20:07:34 +01:00
Michael Eischer
f755233210
Replace usages of ioutil.ReadDir
...
This changes the return type to []fs.DirEntry. However, as we only use
the filenames anyways, this doesn't make a difference.
2022-12-02 19:54:27 +01:00
Michael Eischer
fa20a78bb6
Merge pull request #4056 from greatroar/cleanup
...
backend, fs, options: Minor cleanup
2022-12-02 19:44:54 +01:00
Michael Eischer
ff7ef5007e
Replace most usages of ioutil with the underlying function
...
The ioutil functions are deprecated since Go 1.17 and only wrap another
library function. Thus directly call the underlying function.
This commit only mechanically replaces the function calls.
2022-12-02 19:36:43 +01:00
greatroar
65612d797c
backend, options: Prefer strings.Cut to SplitN
...
Also realigned the various "split into host🪣 prefix"
implementations.
2022-12-02 19:19:14 +01:00
Michael Eischer
2d5e28e777
Merge pull request #4059 from restic/dependabot/go_modules/github.com/minio/minio-go/v7-7.0.45
...
build(deps): bump github.com/minio/minio-go/v7 from 7.0.44 to 7.0.45
2022-12-01 21:20:57 +01:00
dependabot[bot]
4fefa2ade2
build(deps): bump github.com/minio/minio-go/v7 from 7.0.44 to 7.0.45
...
Bumps [github.com/minio/minio-go/v7](https://github.com/minio/minio-go ) from 7.0.44 to 7.0.45.
- [Release notes](https://github.com/minio/minio-go/releases )
- [Commits](https://github.com/minio/minio-go/compare/v7.0.44...v7.0.45 )
---
updated-dependencies:
- dependency-name: github.com/minio/minio-go/v7
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-12-01 01:03:42 +00:00
Michael Eischer
3c5d1eabe9
Merge pull request #4051 from restic/dependabot/go_modules/github.com/klauspost/compress-1.15.12
...
build(deps): bump github.com/klauspost/compress from 1.15.9 to 1.15.12
2022-11-28 21:47:55 +01:00
Michael Eischer
bec391ee26
Merge pull request #4053 from greatroar/xattr
...
Upgrade pkg/xattr to version with Solaris FIFO fix
2022-11-28 21:06:19 +01:00
greatroar
daafcaf380
Upgrade pkg/xattr to version with Solaris FIFO fix
...
This version doesn't have a release tag yet, but it's 0.4.9 + one patch.
Fixes #4003 .
2022-11-28 20:43:51 +01:00
Alexander Neumann
1d7e7fcd6b
Merge pull request #4049 from MichaelEischer/fix-rewrite-docs
...
rewrite: fix link anchors in documentation
2022-11-28 19:39:38 +01:00
dependabot[bot]
57d59c71e3
build(deps): bump github.com/klauspost/compress from 1.15.9 to 1.15.12
...
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress ) from 1.15.9 to 1.15.12.
- [Release notes](https://github.com/klauspost/compress/releases )
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml )
- [Commits](https://github.com/klauspost/compress/compare/v1.15.9...v1.15.12 )
---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-28 18:39:35 +00:00
Alexander Neumann
bb83c78ee5
Merge pull request #4047 from MichaelEischer/clean-ci-configuration
...
Cleanup CI configuration
2022-11-28 19:38:33 +01:00
greatroar
60aa87bbab
fs: Remove explicit type check in extendedStat
...
Without comma-ok, the runtime inserts the same check with a similar
enough panic message:
interface conversion: interface {} is nil, not *syscall.Stat_t
2022-11-27 19:58:06 +01:00
Michael Eischer
34609bca0e
Merge pull request #4050 from greatroar/lruv2
...
bloblru: Upgrade to hashicorp/golang-lru/v2
2022-11-27 17:37:14 +01:00
greatroar
e5d597fd22
bloblru: Upgrade to hashicorp/golang-lru/v2
...
The new genericized LRU cache no longer needs to have the IDs separately
allocated:
name old time/op new time/op delta
Add-8 494ns ± 2% 388ns ± 2% -21.46% (p=0.000 n=10+9)
name old alloc/op new alloc/op delta
Add-8 176B ± 0% 152B ± 0% -13.64% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Add-8 5.00 ± 0% 3.00 ± 0% -40.00% (p=0.000 n=10+10)
2022-11-27 17:18:13 +01:00
Michael Eischer
0eddc89e98
doc: design.rst: Fix highlighting for index snippet
...
JSON does not support comments. As JSON is a subset of Javascript, use
the latter instead.
2022-11-27 17:01:27 +01:00
Michael Eischer
41b0f1d43a
doc: fix link to amazon s3 section
2022-11-27 17:01:22 +01:00
Michael Eischer
6a793db9ca
rewrite: fix link anchors in documentation
2022-11-27 16:38:10 +01:00
Michael Eischer
05cebc1c4b
Merge pull request #4044 from restic/dependabot/go_modules/cloud.google.com/go/storage-1.28.0
...
build(deps): bump cloud.google.com/go/storage from 1.25.0 to 1.28.0
2022-11-27 15:23:06 +01:00
Michael Eischer
ce39727846
Merge pull request #4036 from restic/dependabot/go_modules/github.com/pkg/profile-1.7.0
...
build(deps): bump github.com/pkg/profile from 1.6.0 to 1.7.0
2022-11-27 15:22:16 +01:00
Michael Eischer
9aa06ce959
CI: remove option to configure command used to install go tools
...
With the minimum required go version of 1.18, we always use `go
install`.
2022-11-27 15:07:29 +01:00
Michael Eischer
5968971313
CI: remove dependabot ignore for bazil.org/fuse
...
We've switched to a fork of the original library, thus the ignore is no
longer necessary.
2022-11-27 15:06:30 +01:00
dependabot[bot]
95374767de
build(deps): bump github.com/pkg/profile from 1.6.0 to 1.7.0
...
Bumps [github.com/pkg/profile](https://github.com/pkg/profile ) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/pkg/profile/releases )
- [Commits](https://github.com/pkg/profile/compare/v1.6.0...v1.7.0 )
---
updated-dependencies:
- dependency-name: github.com/pkg/profile
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-27 14:02:32 +00:00
dependabot[bot]
c100a62ebf
build(deps): bump cloud.google.com/go/storage from 1.25.0 to 1.28.0
...
Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go ) from 1.25.0 to 1.28.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases )
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-cloud-go/compare/pubsub/v1.25.0...spanner/v1.28.0 )
---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-27 14:01:36 +00:00
Michael Eischer
c41a1b66e1
Merge pull request #4037 from restic/dependabot/go_modules/google.golang.org/api-0.103.0
...
build(deps): bump google.golang.org/api from 0.93.0 to 0.103.0
2022-11-27 15:00:16 +01:00
dependabot[bot]
705aed0ecb
build(deps): bump google.golang.org/api from 0.93.0 to 0.103.0
...
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.93.0 to 0.103.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.93.0...v0.103.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-27 13:45:08 +00:00
Michael Eischer
28d6de648c
Merge pull request #4040 from restic/dependabot/go_modules/github.com/spf13/cobra-1.6.1
...
build(deps): bump github.com/spf13/cobra from 1.5.0 to 1.6.1
2022-11-27 14:44:25 +01:00
Michael Eischer
bb40b55d1c
Merge pull request #4038 from restic/dependabot/go_modules/github.com/cenkalti/backoff/v4-4.2.0
...
build(deps): bump github.com/cenkalti/backoff/v4 from 4.1.3 to 4.2.0
2022-11-27 14:13:39 +01:00
dependabot[bot]
a24c1e99a6
build(deps): bump github.com/cenkalti/backoff/v4 from 4.1.3 to 4.2.0
...
Bumps [github.com/cenkalti/backoff/v4](https://github.com/cenkalti/backoff ) from 4.1.3 to 4.2.0.
- [Release notes](https://github.com/cenkalti/backoff/releases )
- [Commits](https://github.com/cenkalti/backoff/compare/v4.1.3...v4.2.0 )
---
updated-dependencies:
- dependency-name: github.com/cenkalti/backoff/v4
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-27 12:57:41 +00:00
dependabot[bot]
fd56ead4a8
build(deps): bump github.com/spf13/cobra from 1.5.0 to 1.6.1
...
Bumps [github.com/spf13/cobra](https://github.com/spf13/cobra ) from 1.5.0 to 1.6.1.
- [Release notes](https://github.com/spf13/cobra/releases )
- [Commits](https://github.com/spf13/cobra/compare/v1.5.0...v1.6.1 )
---
updated-dependencies:
- dependency-name: github.com/spf13/cobra
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-27 12:57:23 +00:00
Michael Eischer
cc679c6494
Merge pull request #4041 from MichaelEischer/require-go-1.18
...
Require go 1.18
2022-11-27 13:56:43 +01:00
greatroar
c9c7671c58
fuse: Clean up inode generation
2022-11-27 13:53:42 +01:00
Michael Eischer
530f129a39
rest: remove workaround for content-length handling bug
2022-11-27 13:18:44 +01:00
Michael Eischer
8ad231bcad
bump version numbers in instructions to reproduce binaries
2022-11-27 13:18:44 +01:00
Michael Eischer
a1eb923876
remove no longer necessary conditional compiles
2022-11-27 13:18:44 +01:00
Michael Eischer
bcdfc2a8ea
CI: allow dependabot update of oauth2
...
Our minimum go version is new enough to allow updating the library.
2022-11-27 13:18:44 +01:00
Michael Eischer
686b0b2a3e
update the minimum required go version to 1.18
2022-11-27 13:18:43 +01:00
Michael Eischer
69a2e81bd3
Merge pull request #4039 from restic/dependabot/go_modules/github.com/google/go-cmp-0.5.9
...
build(deps): bump github.com/google/go-cmp from 0.5.8 to 0.5.9
2022-11-26 17:39:27 +01:00
dependabot[bot]
278e93f738
build(deps): bump github.com/google/go-cmp from 0.5.8 to 0.5.9
...
Bumps [github.com/google/go-cmp](https://github.com/google/go-cmp ) from 0.5.8 to 0.5.9.
- [Release notes](https://github.com/google/go-cmp/releases )
- [Commits](https://github.com/google/go-cmp/compare/v0.5.8...v0.5.9 )
---
updated-dependencies:
- dependency-name: github.com/google/go-cmp
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2022-11-26 11:35:05 +00:00
Michael Eischer
747d2ecd7b
Merge pull request #4042 from restic/skip-cloud-test-for-dependabot
...
CI: skip cloud tests for dependabot pull requests
2022-11-26 12:34:16 +01:00
Michael Eischer
98c6ca9d8f
CI: skip cloud tests for dependabot pull requests
2022-11-26 12:23:55 +01:00
Michael Eischer
9113b2620f
Merge pull request #4024 from MichaelEischer/macos-fuse
...
mount: switch to anacrolix fork of bazil/fuse
2022-11-26 12:15:14 +01:00
Michael Eischer
f115d64634
Merge pull request #4022 from MichaelEischer/race-checker
...
CI: Run the golang race checker
2022-11-26 12:13:50 +01:00
Michael Eischer
923c06cea0
Merge pull request #4025 from MichaelEischer/update-minio
...
Update minio library to add `credential_process` support
2022-11-25 23:21:57 +01:00
Michael Eischer
f4d3ed77c4
update minio library
2022-11-25 22:36:21 +01:00
greatroar
189e0fe5a9
fuse: Better inode generation
...
Hard links to the same file now get the same inode within the FUSE
mount. Also, inode generation is faster and, more importantly, no longer
allocates.
Benchmarked on Linux/amd64. Old means the benchmark with
sink = fs.GenerateDynamicInode(1, sub.node.Name)
instead of calling inodeFromNode. Results:
name old time/op new time/op delta
Inode/no_hard_links-8 137ns ± 4% 34ns ± 1% -75.20% (p=0.000 n=10+10)
Inode/hard_link-8 33.6ns ± 1% 9.5ns ± 0% -71.82% (p=0.000 n=9+8)
name old alloc/op new alloc/op delta
Inode/no_hard_links-8 48.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10)
Inode/hard_link-8 0.00B 0.00B ~ (all equal)
name old allocs/op new allocs/op delta
Inode/no_hard_links-8 1.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
Inode/hard_link-8 0.00 0.00 ~ (all equal)
2022-11-16 08:35:01 +01:00
Michael Eischer
32ffcd86a2
Merge pull request #3993 from MichaelEischer/backup-json-full-snapshot-id
...
backup: print full snapshot id in JSON summary
2022-11-12 20:42:35 +01:00
Michael Eischer
f032a9d0ad
prune: report how much data must be repacked to compressed the repo
...
prune now reports the remaining size of pack files containing
uncompressed blobs. The displayed value is suitable for use with `--max-repack-size`.
2022-11-12 20:20:23 +01:00
Michael Eischer
66818a8f98
Merge pull request #3980 from MichaelEischer/prune-compression-stats
...
prune: Correctly count used/duplicate blobs for partially compressed repos
2022-11-12 20:06:56 +01:00
Michael Eischer
4b5234924b
Merge pull request #2875 from fgma/issue2699
...
issue2699: restore symlinks on windows when run as admin user
2022-11-12 20:06:45 +01:00
Michael Eischer
726a1969cd
Merge pull request #2731 from dionorgua/rewrite-snapshot
...
Implement 'rewrite' command to exclude files from existing snapshots
2022-11-12 20:06:35 +01:00
Michael Eischer
bb0fa76c06
Cleanup exclude pattern collection
2022-11-12 19:55:22 +01:00
Michael Eischer
537cfe2e4c
rewrite: Fix check that an exclude pattern was passed
...
The old check did not consider files containing case insensitive
excludes. The check is now implemented as a function of the
excludePatternOptions struct to improve cohesion.
2022-11-12 19:55:22 +01:00
Leo R. Lundgren
f175da2756
rewrite: Polish documentation
2022-11-12 19:55:22 +01:00
Leo R. Lundgren
f86ef4d3dd
rewrite: Polish code and add missing messages
2022-11-12 19:55:22 +01:00
Leo R. Lundgren
c15bedccc0
rewrite: Revert unrelated documentation change
2022-11-12 19:55:22 +01:00
Michael Eischer
f88acd4503
rewrite: Fail if a tree contains an unknown field
...
In principle, the JSON format of Tree objects is extensible without
requiring a format change. In order to not loose information just play
it safe and reject rewriting trees for which we could loose data.
2022-11-12 19:55:22 +01:00
Michael Eischer
11b8c3a158
rewrite: add documentation
2022-11-12 19:55:22 +01:00
Michael Eischer
ec0c91e233
rewrite: Add tests for further ways to use the command
2022-11-12 19:55:22 +01:00
Michael Eischer
0224e276ec
walker: Add tests for FilterTree
2022-11-12 19:55:22 +01:00
Michael Eischer
73f54cc5ea
rewrite: rename --inplace to --forget
2022-11-12 19:55:22 +01:00
Michael Eischer
a47d9a1c40
rewrite: use unified snapshot filter options
2022-11-12 19:55:22 +01:00
Michael Eischer
b044649118
rewrite: add minimal test
2022-11-12 19:55:22 +01:00
Michael Eischer
375a3db64d
rewrite: non-exclusive lock if snapshots are only added
2022-11-12 19:55:22 +01:00
Michael Eischer
327f418a9c
rewrite: cleanup err handling and output
2022-11-12 19:55:22 +01:00
Michael Eischer
ad14d6e4ac
rewrite: use SelectByName like in the backup command
2022-11-12 19:55:22 +01:00
Michael Eischer
7ebaf6e899
rewrite: start repository uploader goroutines
2022-11-12 19:55:22 +01:00
Michael Eischer
559acea0d8
unify exclude pattern options
2022-11-12 19:55:22 +01:00
Michael Eischer
4cace1ffe9
unify exclude patterns with backup command
2022-11-12 19:55:22 +01:00
Michael Eischer
2b69a1c53b
rewrite: filter all snapshots if none are specified
2022-11-12 19:55:22 +01:00
Michael Eischer
f6339b88af
rewrite: extract tree filtering
2022-11-12 19:55:22 +01:00
Michael Eischer
c0f7ba2388
rewrite: simplify dryrun
2022-11-12 19:55:22 +01:00
Michael Eischer
4d6ab83019
rewrite: use treejsonbuilder
2022-11-12 19:55:22 +01:00
Michael Eischer
82592b88b5
rewrite: address most review comments
2022-11-12 19:55:22 +01:00
Michael Eischer
b922774343
rewrite: fix compilation
2022-11-12 19:55:22 +01:00
Dmitry Nezhevenko
dc29709742
Implement 'rewrite' command to exclude files from existing snapshots
2022-11-12 19:55:22 +01:00
Michael Eischer
220eaee76b
mount: switch to anacrolix fork of bazil/fuse
...
The anacrolix fork contains the latest changes from bazil/fuse and
additionally provides support for recent versions of macFUSE.
2022-11-12 19:22:31 +01:00
Michael Eischer
6fa45d0d39
Merge pull request #4011 from greatroar/backup-stdin-password
...
cmd: Don't read password from stdin for backup --stdin
2022-11-12 19:18:56 +01:00
Michael Eischer
bbd180ae21
Merge pull request #4017 from Rajpratik71/Rajpratik71-patch-1
...
feat: dependabot workflow automation for updating dependency
2022-11-12 15:48:48 +01:00
Pratik Raj
bef1064b8e
chore: ignore upgrade for 'bazil/fuse' and 'golang.org/x/oauth2'
2022-11-12 19:39:16 +05:30
Michael Eischer
7b4fe7bad5
Merge pull request #4021 from greatroar/mac-fsync
...
backend/local: Ignore ENOTTY for fsync on Mac
2022-11-11 23:10:37 +01:00
greatroar
348e966daa
backend/local: Ignore ENOTTY for fsync on Mac
...
Fixes #4016 .
2022-11-11 22:51:51 +01:00
Michael Eischer
0e5fe4c6ab
CI: run golang race checker
2022-11-11 22:15:22 +01:00
Michael Eischer
13fbc96ed3
lock: Synchronize `Refresh()` and `Stale()`
...
The lock test creates a lock and checks that it is not stale. However,
it is possible that the lock is refreshed concurrently, which updates
the lock timestamp. Checking the timestamp in `Stale()` without
synchronization results in a data race. Thus add a lock to prevent
concurrent accesses.
2022-11-11 21:52:53 +01:00
Michael Eischer
e1ba7ab684
lock: Don't copy the lock when checking for process existence
...
The lock test creates a lock and checks that it is not stale. This also
tests whether the corresponding process still exists. However, it is
possible that the lock is refreshed concurrently, which updates the lock
timestamp. Calling `processExists()` with a value receiver, however,
creates an unsynchronized copy of this field. Thus call the method using
a pointer receiver.
2022-11-11 21:45:55 +01:00
Michael Eischer
dc060356c2
mount: only start next test after mount command cleanup is complete
...
The test did not wait for the mount command to fully shutdown all
running goroutines. This caused the go race detector to report a data
race related to lock refreshes.
==================
WARNING: DATA RACE
Write at 0x0000021bdfdb by goroutine 667:
github.com/restic/restic/internal/backend/retry.TestFastRetries()
/restic/restic/internal/backend/retry/testing.go:7 +0x18f
github.com/restic/restic/cmd/restic.withTestEnvironment()
/restic/restic/cmd/restic/integration_helpers_test.go:175 +0x183
github.com/restic/restic/cmd/restic.TestMountSameTimestamps()
/restic/restic/cmd/restic/integration_fuse_test.go:202 +0xac
testing.tRunner()
/usr/lib/go/src/testing/testing.go:1446 +0x216
testing.(*T).Run.func1()
/usr/lib/go/src/testing/testing.go:1493 +0x47
Previous read at 0x0000021bdfdb by goroutine 609:
github.com/restic/restic/internal/backend/retry.(*Backend).retry()
/restic/restic/internal/backend/retry/backend_retry.go:72 +0x9e
github.com/restic/restic/internal/backend/retry.(*Backend).Remove()
/restic/restic/internal/backend/retry/backend_retry.go:149 +0x17d
github.com/restic/restic/internal/cache.(*Backend).Remove()
/restic/restic/internal/cache/backend.go:38 +0x11d
github.com/restic/restic/internal/restic.(*Lock).Unlock()
/restic/restic/internal/restic/lock.go:190 +0x249
github.com/restic/restic/cmd/restic.refreshLocks.func1()
/restic/restic/cmd/restic/lock.go:86 +0xae
runtime.deferreturn()
/usr/lib/go/src/runtime/panic.go:476 +0x32
github.com/restic/restic/cmd/restic.lockRepository.func2()
/restic/restic/cmd/restic/lock.go:61 +0x71
[...]
Goroutine 609 (finished) created at:
github.com/restic/restic/cmd/restic.lockRepository()
/restic/restic/cmd/restic/lock.go:61 +0x488
github.com/restic/restic/cmd/restic.lockRepo()
/restic/restic/cmd/restic/lock.go:25 +0x219
github.com/restic/restic/cmd/restic.runMount()
/restic/restic/cmd/restic/cmd_mount.go:126 +0x1f8
github.com/restic/restic/cmd/restic.testRunMount()
/restic/restic/cmd/restic/integration_fuse_test.go:61 +0x1ce
github.com/restic/restic/cmd/restic.checkSnapshots.func1()
/restic/restic/cmd/restic/integration_fuse_test.go:90 +0x124
==================
2022-11-11 21:43:01 +01:00
Michael Eischer
32c9667990
Merge pull request #4019 from MichaelEischer/fix-file-saver-race
...
archiver: Fix race condition resulting in files containing null IDs
2022-11-11 20:52:33 +01:00
Michael Eischer
d268552a0a
Merge pull request #4014 from MichaelEischer/fix-debug-examine
...
debug: fix crash in `debug examine --reupload-blobs`
2022-11-10 20:37:32 +01:00
Michael Eischer
5756c96c9f
archiver: Fix race condition resulting in files containing null IDs
...
In some rare cases files could be created which contain null IDs (all
zero) in their content list. This was caused by a race condition between
growing the `Content` slice and inserting the blob IDs into it. In some
cases the blob ID was written to the old slice, which a short time
afterwards was replaced with a larger copy, that did not yet contain the
blob ID.
2022-11-10 20:19:37 +01:00
Pratik Raj
df614fff26
feat: dependabot workflow automation for updating dependency
...
Signed-off-by: Pratik Raj <Rajpratik71@gmail.com>
2022-11-10 16:02:03 +05:30
Michael Eischer
11a4bb051e
debug: fix crash in `debug examine --reupload-blobs`
2022-11-09 22:13:17 +01:00
Michael Eischer
5f9ac2b165
Merge pull request #4010 from MichaelEischer/file-saver-sanity-check
...
archiver: Check that saved file does not have null IDs in content
2022-11-08 23:07:32 +01:00
Michael Eischer
b1d1202b1d
archiver: Check that saved file does not have null IDs in content
...
Null IDs in the file content indicate that something went wrong. Thus
fails before saving the affected file.
2022-11-08 22:57:41 +01:00
greatroar
5dceadeb72
cmd: Don't read password from stdin for backup --stdin
2022-11-06 14:55:57 +01:00
Michael Eischer
1ccab95bc4
b2: Support file hiding instead of deleting them permanently
...
Automatically fall back to hiding files if not authorized to permanently
delete files. This allows using restic with an append-only application
key with B2. Thus, an attacker cannot directly delete backups with the
API key used by restic.
To use this feature create an application key without the deleteFiles
capability. It is recommended to restrict the key to just one bucket.
For example using the b2 command line tool:
b2 create-key --bucket <bucketName> <keyName> listBuckets,readFiles,writeFiles,listFiles
Suggested-by: Daniel Gröber <dxld@darkboxed.org>
2022-11-05 20:10:45 +01:00
Michael Eischer
24a2e5cab9
Merge pull request #4008 from MichaelEischer/tweak-lock-refresh-test
...
lock: Tweak timeouts for lock refresh test
2022-11-05 10:53:13 +01:00
Michael Eischer
403390479c
Merge pull request #3997 from greatroar/fuse-hash
...
fuse: Better check for whether snapshots changed
2022-11-05 10:52:11 +01:00
Michael Eischer
d29abc1a31
Merge pull request #4007 from MichaelEischer/hide-compression-level-for-v1-repo
...
Only print compression level starting from repository version 2
2022-11-05 10:33:25 +01:00
greatroar
c091e43b33
fuse: Better check for whether snapshots changed
...
We previously checked whether the set of snapshots might have changed
based only on their number, which fails when as many snapshots are
forgotten as are added. Check for the SHA-256 of their id's instead.
2022-11-05 09:32:45 +01:00
Michael Eischer
aaac63da8d
lock: Tweak timeouts for lock refresh test
...
For some reason the test fails from time to time. Increase the timeouts
to hopefully avoid this issue.
2022-11-04 22:48:18 +01:00
Michael Eischer
fd4d23460f
only print compression level starting from repository version 2
2022-11-04 22:40:07 +01:00
Alexander Neumann
8dd95b710e
Merge pull request #3992 from MichaelEischer/err-on-invalid-compression
...
Return error if RESTIC_COMPRESSION env variable is invalid
2022-11-04 19:41:34 +01:00
Alexander Neumann
783b8781a7
Merge pull request #4000 from restic/min-go-version
...
build: Correct checks for minimum Go version
2022-11-04 10:31:02 +01:00
Alexander Neumann
543649f2f2
Merge pull request #4001 from restic/docker-go-version
...
docker: Increase Go version to 1.19
2022-11-04 10:30:11 +01:00
Leo R. Lundgren
0a4cddb34d
docker: Increase Go version to 1.19
2022-11-03 22:59:59 +01:00
Leo R. Lundgren
333c2c6ed4
build: Correct checks for minimum Go version
2022-11-03 22:50:07 +01:00
rawtaz
92df039e5d
Merge pull request #3996 from MichaelEischer/fix-ui-progress
...
backup: fix stuck status bar
2022-11-02 21:48:16 +01:00
Michael Eischer
9354262b1b
backup: fix stuck status bar
...
The status bar got stuck once the first error was reported, the scanner
completed or some file was backed up. Either case sets a flag that the
scanner has started.
This flag is used to hide the progress bar until the flag is set. Due to
an inverted condition, the opposite happened and the status stopped
refreshing once the flag was set.
In addition, the scannerStarted flag was not set when the scanner just
reported progress information.
2022-11-02 21:31:13 +01:00
Michael Eischer
06141ce1f4
backup: print full snapshot id in JSON summary
2022-10-31 19:03:42 +01:00
Michael Eischer
59a90943bb
Merge pull request #3983 from greatroar/formatting
...
Centralize and fix formatting of bytes, percentages, durations
2022-10-31 18:52:24 +01:00
greatroar
5ab3e6276a
ui: Fix FormatBytes at exactly 1024 time a unit
...
1024 would be displayed as "1024 bytes" instead of "1.000 KiB", etc.
2022-10-31 18:39:28 +01:00
rawtaz
4f1fae9c98
Merge pull request #3982 from MichaelEischer/show-compression-mode
...
Show selected compression level when opening repository
2022-10-30 21:29:42 +01:00
Michael Eischer
8fe159cc5a
enable ysmlink tests for windows
2022-10-30 18:43:04 +01:00
Michael Eischer
3499c6354e
Merge pull request #3955 from MichaelEischer/async-futurefile-completion
...
Improve archiver performance for small files
2022-10-30 18:38:04 +01:00
Michael Eischer
144257f8bd
restore symlink timestamps on windows
2022-10-30 11:04:04 +01:00
Michael Eischer
c0f34af9db
backup: hide files from status which are read completely but not saved
...
As the FileSaver is asynchronously waiting for all blobs of a file to be
stored, the number of active files is higher than the number of files
from which restic is reading concurrently. Thus to not confuse users,
only display files in the status from which restic is currently reading.
2022-10-30 10:29:12 +01:00
Michael Eischer
a571fc4aa1
add changelog for faster backups with small files
2022-10-30 10:29:12 +01:00
Michael Eischer
b52a8ff05c
ui: Properly clear lines no longer used for status
...
Previously, the old status text remained until it was overwritten.
2022-10-30 10:29:12 +01:00
Michael Eischer
b4de902596
archiver: Asynchronously complete FutureFile
...
After reading and chunking all data in a file, the FutureFile still has
to wait until the FutureBlobs are completed. This was done synchronously
which results in blocking the file saver and prevents the next file from
being read.
By replacing the FutureBlob with a callback, it becomes possible to
complete the FutureFile asynchronously.
2022-10-30 10:29:11 +01:00
Michael Eischer
47e05080a9
Merge pull request #3990 from MichaelEischer/lock-refresh-test
...
lock: add test to check that refreshing works
2022-10-30 10:15:44 +01:00
Michael Eischer
c7ace314f6
Merge pull request #3989 from greatroar/eachbypack
...
More compact data structure for Index.EachByPack
2022-10-30 00:02:55 +02:00
greatroar
0e8893dae9
index: Compact data structure for Index.EachByPack
2022-10-29 23:09:17 +02:00
greatroar
137f0bc944
repository: Fix benchmarkSaveAndEncrypt
2022-10-29 23:09:17 +02:00
Michael Eischer
01f0db4e56
return error if RESTIC_COMPRESSION env variable is invalid
2022-10-29 22:03:39 +02:00
Michael Eischer
7c87fb941c
Merge pull request #3986 from greatroar/counter
...
ui/progress: Load both values in a single Lock/Unlock
2022-10-29 21:50:55 +02:00
Michael Eischer
3b0bb02a68
Merge pull request #3977 from greatroar/progress
...
ui/backup: Replace channels with a mutex
2022-10-29 21:33:04 +02:00
Michael Eischer
0d260cfd82
enable symlink test on windows
2022-10-29 21:26:34 +02:00
fgma
8e5eb1090c
issue2699: restore symlinks on windows when run as admin user
2022-10-29 21:19:33 +02:00
rawtaz
af3f7c866f
Merge pull request #3988 from FelixBurkhard/FelixBurkhard-patch-1
...
Clearify what Azure account name means
2022-10-29 13:32:44 +02:00
Michael Eischer
24267e9a9d
lock: add test to check that refreshing works
2022-10-29 11:26:00 +02:00
Michael Eischer
8e51e1e605
shorten 'repository opened' output
2022-10-29 11:22:00 +02:00
FelixBurkhard
575d26ec87
Clearify what Azure account name means
...
When reading it first it was not clear to me the 'account name' meant the name of the
Azure Storage Account and not the Azure account itself.
2022-10-29 00:27:43 +02:00
greatroar
2dafda9164
ui/progress: Load both values in a single Lock/Unlock
...
We always need both values, except in a test, so we don't need to lock
twice and risk scheduling in between.
Also, removed the resetting in Done. This copied a mutex, which isn't
allowed. Static analyzers tend to trip over that.
2022-10-25 07:55:24 +02:00
Michael Eischer
f8910bc4ff
Merge pull request #3985 from saltsa/fix_lock_refresh
...
Fix bug in lock refresh monitoring
2022-10-24 22:59:18 +02:00
Joonas Aunola
b06427c9f6
fix Unix to UnixNano
2022-10-23 23:40:21 +03:00
greatroar
006380199e
cmd, ui: Deduplicate formatting utilities
2022-10-23 13:40:07 +02:00
greatroar
04216eb9aa
ui/backup: Replace channels with a mutex
...
The channel-based algorithm had grown quite complicated. This is easier
to reason about and likely to be more performant with very many
CompleteBlob calls.
2022-10-23 13:28:41 +02:00
Michael Eischer
4fea3a413d
show selected compression level when opening repository
2022-10-22 20:18:46 +02:00
Michael Eischer
ba58ccbe07
prune: add remark about non-deterministic blob selection
2022-10-22 19:46:10 +02:00
Michael Eischer
05651d6d4f
prune: Correctly count used/duplicate blobs for partially compressed repos
...
Counting the first occurrence of a duplicate blob as used and counting
all other as duplicates, independent of which instance of the blob is
kept, is only accurate if all copies of the blob have the same size. This
is no longer the case for a repository containing both compressed and
uncompressed blobs.
Thus for duplicated blobs first count all instances as duplicates and
then subtract the actually used instance later on.
2022-10-22 19:24:36 +02:00
Michael Eischer
b57d42905c
Merge pull request #3899 from MichaelEischer/less-prune-mem
...
Optimize prune memory usage
2022-10-22 18:56:02 +02:00
Michael Eischer
d966c52707
prune: allow gc of set of repacked blobs before index rebuild
2022-10-22 18:45:12 +02:00
Michael Eischer
1e2794fa55
add prune memory optimization changelog
2022-10-22 18:45:12 +02:00
Michael Eischer
68c9cb9c6a
prune: Shrink keepBlobs set if possible
...
As long as only a small fraction of the data in a repository is
rewritten, the keepBlobs set will be rather small after cleaning it up.
As golang maps do not shrink their memory usage, just copy the contents
over to a new map. However, only copy the map if the cleanup removed at
least half the entries.
2022-10-22 18:45:12 +02:00
Michael Eischer
c4fc5c97f9
prune: Use a single CountedBlobSet to track blobs
...
The set covers necessary, existing and duplicate blobs. This removes the
duplicate sets used to track whether all necessary blobs also exist.
This reduces the memory usage of prune by about 20-30%.
2022-10-22 18:45:12 +02:00
Michael Eischer
b21241ec1c
restic: Add CountedBlobSet type
...
This allows maintaining a usage counter for each blob.
2022-10-22 18:45:12 +02:00
Michael Eischer
ee6688a9f6
Merge pull request #3915 from plumbeo/compression-stats
...
restic stats: print uncompressed size in mode raw-data
2022-10-21 22:10:29 +02:00
Michael Eischer
27634a1a68
Merge pull request #3978 from MichaelEischer/fix-negative-pattern-example
...
Remove misleading wildcard from negative exclude pattern example
2022-10-21 22:04:30 +02:00
Michael Eischer
aa77702e49
Merge pull request #3971 from MichaelEischer/parallel-list
...
Unify ForAllIndex/Snapshot/Lock functions
2022-10-21 21:58:33 +02:00
Michael Eischer
6877aaa8aa
Merge pull request #3967 from MichaelEischer/archiver-extract-exclude-options
...
backup: extract exclude pattern options
2022-10-21 21:50:00 +02:00
Michael Eischer
2e9ee8577a
Merge pull request #3970 from MichaelEischer/split-retry-backend
...
Split backend package into smaller parts
2022-10-21 21:49:46 +02:00
Michael Eischer
59d46bb3f5
backup: extract exclude pattern options
...
This is a preparation to make the exclude options usable for the
upcoming `rewrite` command.
2022-10-21 21:40:59 +02:00
Michael Eischer
5c7a9a739a
backend: Split RetryBackend into own package
...
The RetryBackend tests depend on the mock backend. When the Backend
interface is eventually split from the restic package, this will lead to
a dependency cycle between backend and backend/mock. Thus split the
RetryBackend into a separate package to avoid this problem.
2022-10-21 21:38:17 +02:00
Michael Eischer
32603d49c4
backend: remove unused ErrorBackend
2022-10-21 21:36:05 +02:00
Michael Eischer
8c18c65b3b
backend: remove unused Paths variable
2022-10-21 21:36:05 +02:00
Michael Eischer
4ccd5e806b
backend: split layout code into own subpackage
2022-10-21 21:36:05 +02:00
Michael Eischer
b361284f28
Merge pull request #3979 from MichaelEischer/backup-less-time-now
...
backup: reduce calls to time.Now
2022-10-21 21:33:34 +02:00
Michael Eischer
738b2a0445
parallelize more List usages
2022-10-21 21:26:45 +02:00
Michael Eischer
ae45f3b04f
restic: Unify code to load Index/Lock/Snapshot
2022-10-21 21:25:11 +02:00
Michael Eischer
8e2695be0b
Merge pull request #3973 from MichaelEischer/speedup-integration-tests
...
speed-up integration tests by reducing the RetryBackend timeout
2022-10-21 21:17:35 +02:00
Michael Eischer
35d968bcde
Merge pull request #3969 from MichaelEischer/key-by-id
...
Port restic.Find to return IDs and identify keys by restic.ID
2022-10-21 21:15:40 +02:00
Michael Eischer
4133fee6f9
Merge pull request #3972 from MichaelEischer/fix-flaky-lock-cancel-test
...
lock: fix flaky TestLockFailedRefresh
2022-10-21 21:12:34 +02:00
Michael Eischer
c8c8391b21
Merge pull request #3974 from greatroar/cleanup
...
More cleanups and a micro-optimization
2022-10-21 21:11:37 +02:00
Michael Eischer
ee7c28f5e6
backup: reduce calls to time.Now
...
Archiver.Save queries the current time multiple times. This commit
removes one of these calls as they showed up while profiling a backup of
a nearly unchanged dataset containing 3 million files.
2022-10-21 20:55:01 +02:00
Michael Eischer
3e60d38a23
Remove misleading wildcard from negative exclude pattern example
...
There is no need to use a special wildcard `**` to demonstrate negative
patterns. Actually, it is both slower than the simpler variant and seems
to confuse users.
2022-10-21 20:48:45 +02:00
greatroar
9adae5521d
cache: Call interface method once
2022-10-21 14:32:46 +02:00
greatroar
201e5c7e74
backup: Clean up progress reporting code
2022-10-21 13:48:30 +02:00
plumbeo
a6f83e0011
Add changelog
2022-10-17 15:38:42 +02:00
plumbeo
bc945d0bf0
restic stats: add more compression statistics
...
Calculate and display compression ratio, space saving and progress
2022-10-17 15:38:38 +02:00
greatroar
b513597546
internal/restic: Make FileType a uint8 instead of a string
...
The string form was presumably useful before the introduction of
layouts, but right now it just makes call sequences and garbage
collection more expensive (the latter because every string contains
a pointer to be scanned).
2022-10-16 10:59:01 +02:00
greatroar
22147e1e02
all: Minor cleanups
...
if x { return true } return false => return x
fmt.Sprintf("%v", x) => fmt.Sprint(x) or x.String()
The fmt.Sprintf idiom is still used in the SecretString tests, where it
serves security hardening.
2022-10-16 10:50:39 +02:00
greatroar
d03460010f
internal/restic: Fix ID.UnmarshalJSON, ParseID
...
ID.UnmarshalJSON accepted non-JSON input with ' as the string delimiter.
Also, the error message for non-hex input was less informative than it
could be and it performed too many checks.
Changed ParseID to keep the error messages consistent.
2022-10-16 10:39:52 +02:00
Michael Eischer
aa39bf3cf6
backend/test: remove duplicate test
...
The test is identical to the tests for the mem backend.
2022-10-15 23:15:07 +02:00
Michael Eischer
28e1c4574b
mem: use cheaper hash for backend
2022-10-15 23:14:33 +02:00
Michael Eischer
c3400d3c55
backend: speedup RetryBackend tests
2022-10-15 23:13:44 +02:00
Michael Eischer
99547518cd
lock: fix flaky TestLockFailedRefresh
...
The comparison of the current time and the last lock refresh were using
seconds represented as integers. As the test only waits for up to one
second, the associated number truncation can cause the test to take
longer than once second and thus to fail.
Switch to nanoseconds to avoid this problem. This also slightly speeds
up the test.
2022-10-15 22:36:32 +02:00
Michael Eischer
e10420553b
speed-up integration tests by reducing the RetryBackend timeout
...
On my machine this decreases the runtime for `./cmd/restic` from 9.5s to
6.5s.
2022-10-15 22:29:58 +02:00
Michael Eischer
367f35db27
Merge pull request #3968 from MichaelEischer/cleanup-complete-blob
...
backup: Remove unused filename parameter from CompleteBlob callback
2022-10-15 16:11:16 +02:00
Michael Eischer
8d62a7adb4
identify keys by ID and not name
2022-10-15 16:07:43 +02:00
Michael Eischer
02634dce7a
restic: change Find to return ids
...
That way consumers no longer have to manually convert the returned name
to an id.
2022-10-15 16:06:54 +02:00
Michael Eischer
964977677f
backup: Remove unused filename parameter from CompleteBlob callback
2022-10-15 15:21:17 +02:00
Michael Eischer
258b487d8f
Merge pull request #3951 from MichaelEischer/rework-snapshot-filter
...
Rework snapshot filtering
2022-10-15 14:47:47 +02:00
Michael Eischer
de9bc031df
add changelog for ls handling of missing snapshots
2022-10-15 13:34:50 +02:00
Michael Eischer
246d3032ae
restic: Don't list snapshots if FindSnapshot gets full id
2022-10-15 13:34:34 +02:00
Michael Eischer
d8c00b9726
add comment
2022-10-15 13:34:21 +02:00
Michael Eischer
a3113c6097
restic: Change FindSnapshot functions to return the snapshot
2022-10-15 13:34:04 +02:00
Michael Eischer
b50f48594d
restic: cleanup arguments of findLatestSnapshot
2022-10-15 13:33:48 +02:00
Michael Eischer
61e827ae4f
restic: hide findLatestSnapshot
2022-10-15 13:33:32 +02:00
Michael Eischer
fcad5e6f5d
backup: use unified FindFilteredSnapshot
2022-10-15 13:33:29 +02:00
Michael Eischer
0aa73bbd39
ls: proper error handling for non-existent snapshot
...
Use restic.FindFilteredSnapshot to resolve the snapshot ID. This ensures
consistent behavior for all commands using initSingleSnapshotFilterOptions.
2022-10-15 13:32:00 +02:00
Michael Eischer
a81f0432e9
restic: Add unified method to resolve a single snapshot
2022-10-15 13:31:45 +02:00
Michael Eischer
95a1bb4261
restic: Rework error handling of FindFilteredSnapshots and handle snapshotIDs
...
FindFilteredSnapshots no longer prints errors during snapshot loading on
stderr, but instead passes the error to the callback to allow the caller
to decide on what to do.
In addition, it moves the logic to handle an explicit snapshot list from
the main package to restic.
2022-10-15 13:31:26 +02:00
Michael Eischer
cff22a5f01
dump: use correct help text for filter options
2022-10-15 13:31:10 +02:00
Michael Eischer
7a6dcb4831
Merge pull request #3966 from MichaelEischer/cleanup-walker-test
...
walker: Convert tests to use TreeJSONBuilder
2022-10-15 11:25:11 +02:00
Michael Eischer
7cf042118f
walker: Convert tests to use TreeJSONBuilder
...
The old code marshalled the tree blobs different than other places in
restic. The hashed tree blob did not contain a final newline character.
2022-10-15 11:04:13 +02:00
Michael Eischer
cea7191995
Merge pull request #3959 from MichaelEischer/buffered-backup-progress
...
backup: Use buffered channels to collect backup status
2022-10-15 10:57:19 +02:00
Michael Eischer
ba688aad20
Merge pull request #3961 from greatroar/cleanup
...
Misc. cleanup
2022-10-14 21:49:35 +02:00
Michael Eischer
9c290a8093
Merge pull request #3960 from greatroar/errors
...
errors: Drop WithMessage
2022-10-14 21:41:28 +02:00
greatroar
0e155fd9a6
internal/restic: Fix UID/GID parsing
...
The helper function uidGidInt used strconv.ParseInt instead of
ParseUint, so it silently ignored some invalid user/group IDs.
Also, improve the error message. "Invalid UID" is more informative than
having "ParseInt" twice (*strconv.NumError displays the function name).
Finally, the user.User struct can be passed by pointer to get reduce
code size.
2022-10-14 18:21:00 +02:00
greatroar
e0b743c64d
internal/restic: Remove unused ID.EqualString
2022-10-14 18:20:11 +02:00
greatroar
6922360179
ui/backup: Remove unused ProgressReporter type, Progress field
2022-10-14 14:36:19 +02:00
greatroar
d4aadfa389
all: Drop ctxhttp
...
This package is no longer needed, since we can use the stdlib's
http.NewRequestWithContext.
backend/rclone already did, but it needed a different error check due to
a difference between net/http and ctxhttp.
Also, store the http.Client by value in the REST backend (changed to a
pointer when ctxhttp was introduced) and use errors.WithStack instead
of errors.Wrap where the message was no longer accurate. Errors from
http.NewRequestWithContext will start with "net/http" or "net/url", so
they're easy to identify.
2022-10-14 14:33:49 +02:00
greatroar
16849d5361
internal/archiver: Missing argument to errors.Errorf
2022-10-14 14:18:52 +02:00
greatroar
09c14f33c8
internal/checker: Pass Error.Error pointer receiver
2022-10-14 14:13:32 +02:00
greatroar
feb790f497
internal/restic: Use errors.New when no formatting is needed
2022-10-14 14:07:20 +02:00
greatroar
ba44666704
errors: Drop WithMessage
2022-10-14 14:06:47 +02:00
Michael Eischer
1a6160d152
Merge pull request #3880 from MichaelEischer/archiver-savedir-cleanup
...
archiver: Improve handling of "file xxx already present" error
2022-10-08 21:48:14 +02:00
Michael Eischer
21b1d7a880
Merge pull request #3948 from MichaelEischer/split-index
...
repository: split index into a separate package
2022-10-08 21:41:57 +02:00
Michael Eischer
5278ab51c8
archiver: Check that duplicates are only ignored if identical
2022-10-08 21:38:36 +02:00
Michael Eischer
403b01b788
backup: Only return a warning for duplicate directory entries
...
The backup command failed if a directory contains duplicate entries.
Downgrade the severity of this problem from fatal error to a warning.
This allows users to still create a backup.
2022-10-08 21:38:21 +02:00
Michael Eischer
d7d7b4ab27
archiver: refactor TreeSaverTest
2022-10-08 21:29:32 +02:00
Michael Eischer
8e38c43c27
archiver: let FutureNode.Take return an error if no data is available
...
This ensures that we cannot accidentally store an invalid node.
2022-10-08 21:28:39 +02:00
Michael Eischer
2b88cd6eab
archiver: Restructure SaveTree to work like SaveDir
...
SaveTree did not use the TreeSaver but rather managed the tree
collection and upload itself. This prevents using the parallelism
offered by the TreeSaver and duplicates all related code. Using the
TreeSaver can provide some speed-ups as all steps within the backup tree
now rely on FutureNodes. This can be especially relevant for backups
with large amounts of explicitly specified files.
The main difference between SaveTree and SaveDir is, that only the
former can save tree blobs in which nodes have a different name than the
actual file on disk. This is the result of resolving name conflicts
between multiple files with the same name. The filename that must be
used within the snapshot is now passed directly to
restic.NodeFromFileInfo. This ensures that a FutureNode already contains
the correct filename.
2022-10-08 21:28:39 +02:00
Michael Eischer
2e3f1c08c5
repository: split index into a separate package
2022-10-08 21:15:34 +02:00
Michael Eischer
5760ba6989
Merge pull request #3949 from MichaelEischer/simplify-mixedpacks
...
repository: remove IsMixedPack and add replacement for checker
2022-10-08 21:14:14 +02:00
Michael Eischer
5ee25e669a
Merge pull request #3940 from MichaelEischer/better-rclone-error
...
Better error message if connection to rclone fails
2022-10-08 21:14:00 +02:00
Michael Eischer
5600f11696
rclone: Fix stderr handling if command exits unexpectedly
...
According to the documentation of exec.Cmd Wait() must not be called
before completing all reads from the pipe returned by StdErrPipe(). Thus
return a context that is canceled once rclone has exited and use that as
a precondition to calling Wait(). This should ensure that all errors
printed to stderr have been copied first.
2022-10-08 20:16:06 +02:00
Michael Eischer
b8acad4da0
rclone: return rclone error instead of canceled context
...
When rclone fails during the connection setup this currently often
results in a context canceled error. Replace this error with the exit
code from rclone.
2022-10-08 20:15:24 +02:00
Michael Eischer
d3ebec8f21
backup: Use buffered channels to collect backup status
...
When backing up many small files, the unbuffered channels frequently
cause the FileSaver to block when reporting progress information. Thus,
add buffers to these channels to avoid unnecessary scheduling.
As the status information is purely informational, it doesn't matter
that the status reporting shutdown is somewhat racy and could miss a few
final updates.
2022-10-08 18:20:41 +02:00
Michael Eischer
f9d4e0c2af
Merge pull request #3958 from greatroar/errors
...
errors: Drop Cause in favor of Go 1.13 error handling
2022-10-08 18:06:35 +02:00
Michael Eischer
119e6aee01
Merge pull request #3957 from greatroar/typo
...
cmd: Typo in --read-concurrency description
2022-10-08 14:41:35 +02:00
greatroar
07e5c38361
errors: Drop Cause in favor of Go 1.13 error handling
...
The only use cases in the code were in errors.IsFatal, backend/b2,
which needs a workaround, and backend.ParseLayout. The last of these
requires all backends to implement error unwrapping in IsNotExist.
All backends except gs already did that.
2022-10-08 13:08:08 +02:00
greatroar
4eae4d3e1a
cmd: Typo in --read-concurrency description
2022-10-08 11:27:39 +02:00
Michael Eischer
83cb58b4f3
Merge pull request #3956 from MichaelEischer/fix-lock-refresh
...
lock: Use the correct duration to check for expired locks
2022-10-07 22:58:10 +02:00
Michael Eischer
7c5d63a794
lock: Use the correct duration to check for expired locks
2022-10-07 22:39:53 +02:00
Michael Eischer
8b7c952f17
Merge pull request #3953 from keachi/typo
...
Fix typo
2022-10-07 22:18:32 +02:00
Michael Eischer
e43d2d45f7
Merge pull request #3952 from hoelzro/master
...
Update copy documentation to use --from-repo option
2022-10-07 22:18:06 +02:00
Rob Hoelz
03e9a26018
Update copy documentation to use --from-repo option
...
Removing the last references to the deprecated --repo2 option
2022-10-07 22:00:12 +02:00
tr
43cc01d63e
doc: Fix typo
2022-10-05 21:03:14 +02:00
Michael Eischer
7112a132c3
Merge pull request #3950 from MichaelEischer/misc-cleanups
...
Cleanups for cmd_debug/repository and remove dead code from restic package
2022-10-03 12:46:32 +02:00
Michael Eischer
4bb5240720
repository: remove unused PrefixLength
2022-10-03 12:15:53 +02:00
Michael Eischer
999fe29976
repository: hide prepareCache
2022-10-03 12:15:53 +02:00
Michael Eischer
9197c63007
debug: use repository.ListPack wrapper
2022-10-03 12:09:08 +02:00
Michael Eischer
ddcf549eba
repository: remove IsMixedPack and add replacement for checker
...
Repositories with mixed packs are probably quite rare by now. When
loading data blobs from a mixed pack file, this will no longer trigger
caching that file. However, usually tree blobs are accessed first such
that this shouldn't make much of a difference.
The checker gets a simpler replacement.
2022-10-03 12:03:59 +02:00
Michael Eischer
a61fbd287a
Merge pull request #3569 from MichaelEischer/strict-locking
...
Strict repository lock handling
2022-10-03 00:44:44 +02:00
Michael Eischer
6d2d297215
pass global context through cobra
2022-10-03 00:19:46 +02:00
Michael Eischer
49126796d0
lock: fix timer expiry monitoring during standby
...
Monotonic timers are paused during standby. Thus these timers won't fire
after waking up. Fall back to periodic polling to detect too large clock
jumps. See https://github.com/golang/go/issues/35012 for a discussion of
go timers during standby.
2022-10-03 00:19:46 +02:00
Michael Eischer
401e432e9d
lock: Do not ignore invalid lock files
...
While searching for lock file from concurrently running restic
instances, restic ignored unreadable lock files. These can either be
in fact invalid or just be temporarily unreadable. As it is not really
possible to differentiate between both cases, just err on the side of
caution and consider the repository as already locked.
The code retries searching for other locks up to three times to smooth
out temporarily unreadable lock files.
2022-10-03 00:19:46 +02:00
Michael Eischer
aeed420e1a
add changelog
2022-10-03 00:19:46 +02:00
Michael Eischer
9959190e39
lock: Add integration test
...
The tests check that the wrapped context is properly canceled whenever
the repository is unlock or when the lock refresh fails.
2022-10-03 00:19:46 +02:00
Michael Eischer
c3538b063a
lock: Use repository interface instead of struct
2022-10-03 00:19:46 +02:00
Michael Eischer
d92957dd78
lock: Implement strict lock expiry monitoring
...
Restic continued e.g. a backup task even when it failed to renew the
lock or failed to do so in time. For example if a backup client enters
standby during the backup this can allow other operations like `prune`
to run in the meantime (after calling `unlock`). After leaving standby
the backup client will continue its backup and upload indexes which
refer pack files that were removed in the meantime.
This commit introduces a goroutine explicitly monitoring for locks that
are not refreshed in time. To simplify the implementation there's now a
separate goroutine to refresh the lock and monitor for timeouts for each
lock. The monitoring goroutine would now cause the backup to fail as the
client has lost it's lock in the meantime.
The lock refresh goroutines are bound to the context used to lock the
repository initially. The context returned by `lockRepo` is also
cancelled when any of the goroutines exits. This ensures that the
context is cancelled whenever for any reason the lock is no longer
refreshed.
2022-10-03 00:19:46 +02:00
Michael Eischer
928914f821
Prepare for context bound to lock lifetime
2022-10-03 00:19:46 +02:00
Michael Eischer
985722b102
Remove ctx from globalOptions
...
Previously the global context was either accessed via gopts.ctx,
stored in a local variable and then used within that function or
sometimes both. This makes it very hard to follow which ctx or a wrapped
version of it reaches which method.
Thus just drop the context from the globalOptions struct and pass it
explicitly to every command line handler method.
2022-10-03 00:19:46 +02:00
Michael Eischer
ab819b2344
key: Cleanup method signatures
2022-10-03 00:19:46 +02:00
Michael Eischer
d0668b695d
Remove unnecessary context.WithCancel calls
...
The gopts.ctx is cancelled when the main() method of restic exits.
2022-10-03 00:19:46 +02:00
Michael Eischer
7ce4cb7908
Merge pull request #3947 from MichaelEischer/fix-cache-verify-test
...
cache: Fix file descriptor leak in TestBackendRemoveBroken
2022-10-03 00:19:26 +02:00
Michael Eischer
430ab32941
cache: Fix file descriptor leak in TestBackendRemoveBroken
2022-10-03 00:06:44 +02:00
Michael Eischer
e99ad39b34
Merge pull request #2750 from metalsp0rk/min-packsize
...
Add `backup --file-read-concurrency` flag
2022-10-02 23:11:47 +02:00
Michael Eischer
2e606ca70b
backup: rework read concurrency
2022-10-02 22:55:14 +02:00
Kyle Brennan
4a501d7118
backup: add option for file read concurrency
2022-10-02 22:51:45 +02:00
Michael Eischer
9ec7eee803
Merge pull request #3521 from MichaelEischer/redownload-broken-files
...
Redownload files with wrong hash
2022-10-02 22:50:03 +02:00
Michael Eischer
b25d0773b6
Merge pull request #3944 from MichaelEischer/fix-linter-errors
...
CI: ignore warning about missing package comment
2022-09-27 21:41:55 +02:00
Michael Eischer
5265550ff3
CI: ignore warning about missing package comment
2022-09-27 21:31:37 +02:00
Michael Eischer
e89fc2a29d
Merge pull request #3943 from MichaelEischer/find-match-only-valid-ids
...
ignore filenames which are not IDs when expanding a prefix
2022-09-27 20:56:48 +02:00
Michael Eischer
67e4620cd6
Merge pull request #3938 from restic/errdot
...
rclone/sftp: Improve handling of ErrDot errors
2022-09-27 20:33:42 +02:00
Michael Eischer
5d3c5b9e50
restic: ignore filenames which are not IDs when expanding a prefix
...
Some backends generate additional files for each existing file, e.g.
1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef
1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef.sha256
For some commands this leads to an "multiple IDs with prefix" error when
trying to reference a snapshot.
2022-09-27 20:30:40 +02:00
Leo R. Lundgren
ebe9f2c969
rclone/sftp: Improve handling of ErrDot errors
...
Restic now yields a more informative error message when exec.ErrDot occurs.
2022-09-25 16:19:03 +02:00
Michael Eischer
d114e483c4
Add changelog for corrupt data downloads
2022-09-25 11:55:09 +02:00
Michael Eischer
34c1a83340
cache: Drop cache entry if it cannot be processed
...
Failing to process data requested from the cache usually indicates a
problem with the returned data. Assume that the cache entry is somehow
damaged and retry downloading it once.
2022-09-25 11:55:09 +02:00
Michael Eischer
aa3b1925b4
cache: Simplify loadFromCacheOrDelegate
2022-09-25 11:35:35 +02:00
Michael Eischer
5c6b6edefe
retry index, lock and snapshot loading on hash mismatch
2022-09-25 11:35:35 +02:00
Michael Eischer
822422ef03
retry key loading on hash mismatch
2022-09-25 11:35:35 +02:00
Michael Eischer
d6575f53ca
Merge pull request #3942 from MichaelEischer/split-cross-compile-test
...
Split cross compile test
2022-09-24 22:27:08 +02:00
Michael Eischer
78d2312ee9
Merge pull request #3854 from MichaelEischer/sparsefiles
...
restore: Add support for sparse files
2022-09-24 22:04:02 +02:00
Michael Eischer
46b30b9826
split cross compilation into three parts
...
The cross compilation tasks are currently the slowest part of the CI
runs. Splitting it into three parts should reduce its time to roughly
that of the windows CI run.
2022-09-24 22:00:25 +02:00
Michael Eischer
bd191ec60b
update golang-ci to version 1.49
2022-09-24 22:00:08 +02:00
Michael Eischer
519059cca4
update ci actions
2022-09-24 21:59:36 +02:00
Michael Eischer
19afad8a09
restore: support sparse restores also on windows
2022-09-24 21:39:39 +02:00
Michael Eischer
0f89f443c7
update sparse restore changelog
2022-09-24 21:39:39 +02:00
Michael Eischer
c147422ba5
repository: special case SaveBlob for all zero chunks
...
Sparse files contain large regions containing only zero bytes. Checking
that a blob only contains zeros is possible with over 100GB/s for modern
x86 CPUs. Calculating sha256 hashes is only possible with 500MB/s (or
2GB/s using hardware acceleration). Thus we can speed up the hash
calculation for all zero blobs (which always have length
chunker.MinSize) by checking for zero bytes and then using the
precomputed hash.
The all zeros check is only performed for blobs with the minimal chunk
size, and thus should add no overhead most of the time. For chunks which
are not all zero but have the minimal chunks size, the overhead will be
below 2% based on the above performance numbers.
This allows reading sparse sections of files as fast as the kernel can
return data to us. On my system using BTRFS this resulted in about
4GB/s.
2022-09-24 21:39:39 +02:00
Michael Eischer
34fe1362da
restorer: move zeroPrefixLen to restic package
2022-09-24 21:39:39 +02:00
Michael Eischer
a5ebd5de4b
restorer: Fix race condition in partialFile.WriteAt
...
The restorer can issue multiple calls to WriteAt in parallel. This can
result in unexpected orderings of the Truncate and WriteAt calls and
sometimes too short restored files.
2022-09-24 21:39:39 +02:00
Michael Eischer
5b6a77058a
Enable sparseness only conditionally
...
We can either preallocate storage for a file or sparsify it. This
detects a pack file as sparse if it contains an all zero block or
consists of only one block. As the file sparsification is just an
approximation, hide it behind a `--sparse` parameter.
2022-09-24 21:20:00 +02:00
greatroar
3047bf611c
Changelog entry for sparse file restoring
2022-09-24 21:18:48 +02:00
greatroar
5d4568d393
Write sparse files in restorer
...
This writes files by using (*os.File).Truncate, which resolves to the
truncate system call on Unix.
Compared to the naive loop,
for _, b := range p {
if b != 0 {
return false
}
}
the optimized allZero is about 10× faster:
name old time/op new time/op delta
AllZero-8 1.09ms ± 1% 0.09ms ± 1% -92.10% (p=0.000 n=10+10)
name old speed new speed delta
AllZero-8 3.84GB/s ± 1% 48.59GB/s ± 1% +1166.51% (p=0.000 n=10+10)
2022-09-24 21:18:48 +02:00
Michael Eischer
eb83402d39
Merge pull request #3935 from miles170/master
...
Only display the message if there were locks to be removed
2022-09-24 20:53:13 +02:00
Michael Eischer
ef58ddd7b1
Merge pull request #3923 from MichaelEischer/fix-flaky-cache-test
...
cache: fix flaky TestFileSaveConcurrent on windows
2022-09-24 20:52:55 +02:00
Michael Eischer
7fc178aaf4
internal/cache: extend description of cache sharing test failure
2022-09-24 13:07:01 +02:00
Miles Liu
1acbda18f8
Only display the message if there were locks to be removed
...
`restic unlock` now only shows `successfully removed locks` if there were locks to be removed.
In addition, it also reports the number of the removed lock files.
2022-09-24 19:02:24 +08:00
Michael Eischer
da1a359c8b
Merge pull request #3927 from MichaelEischer/faster-index-each
...
Speed up MasterIndex.Each
2022-09-24 12:35:23 +02:00
Michael Eischer
041a51512a
Merge pull request #3780 from jkmw/fix/2578
...
Remove existing path before restoring a symlink
2022-09-24 12:34:42 +02:00
Michael Eischer
1ebd57247a
repository: optimize MasterIndex.Each
...
Sending data through a channel at very high frequency is extremely
inefficient. Thus use simple callbacks instead of channels.
> name old time/op new time/op delta
> MasterIndexEach-16 6.68s ±24% 0.96s ± 2% -85.64% (p=0.008 n=5+5)
2022-09-24 12:21:59 +02:00
Michael Eischer
825b95e313
repository: add benchmark for MasterIndex.Each
2022-09-24 12:21:59 +02:00
greatroar
1220fe9650
internal/cache: Concurrent use of cache not working on Windows
2022-09-17 19:49:44 +02:00
Jerome Küttner
ef618bdd3f
use os.Remove if path already exists on symlink restore
2022-09-14 08:14:31 +02:00
Michael Eischer
b48766d7b8
Merge pull request #3928 from restic/rawtaz-doc-b2-s3
...
doc: Clarify S3 recommendation for B2 slightly
2022-09-13 20:59:50 +02:00
rawtaz
20f1913ef7
doc: Clarify S3 recommendation for B2 slightly
...
This gives slightly more background to the recommendation for using restic's S3 backend with Backblaze B2.
2022-09-12 17:48:59 +02:00
rawtaz
d79e61ce5d
Merge pull request #3925 from hgraeber/add-powershell-completion
...
Add powershell completion
2022-09-11 01:04:57 +02:00
Herbert Graeber
988b386e8b
Add powershell completion
...
- Add code for powersehll complition available in cobra
- Add documentation for powershell completion
- Add changelog for pr3925
2022-09-11 00:44:12 +02:00
rawtaz
14d09a6081
Merge pull request #3912 from MichaelEischer/cleanup-snapshot-filter-options
...
Clean up snapshot filter options
2022-09-11 00:18:42 +02:00
Michael Eischer
381da0443a
tweak snapshot filter descriptions
2022-09-10 23:50:20 +02:00
Michael Eischer
8b9778d537
Merge pull request #3900 from MichaelEischer/b2-init-timeout
...
Add timeout for the initial connection to B2
2022-09-10 23:28:59 +02:00
Michael Eischer
17c27400f8
Merge pull request #3921 from MichaelEischer/filter-cleanup-error-handling
...
filter: deduplicate error handling for pattern validation
2022-09-10 23:24:50 +02:00
Michael Eischer
f76643bd2e
Merge pull request #3894 from MichaelEischer/filter-mount-exit-code
...
Mount should return exit code 0 after pressing Ctrl-C
2022-09-10 23:22:01 +02:00
Michael Eischer
be9ccc186e
Merge pull request #3875 from MichaelEischer/fix-fuse-context-cancel
...
mount: Fix input/output errors for canceled syscalls
2022-09-10 23:20:29 +02:00
Michael Eischer
2363e5c083
Merge pull request #3913 from MichaelEischer/better-migrate-error-message
...
migrate: Report why an migration cannot be applied
2022-09-09 23:37:25 +02:00
Michael Eischer
8e0ca80547
filter: deduplicate error handling for pattern validation
2022-09-09 23:12:41 +02:00
plumbeo
d66e755ac7
Change uncompressed size calculation to account for the encryption overhead
2022-09-08 10:15:19 +02:00
plumbeo
837b816358
restic stats: print uncompressed size in mode raw-data
2022-09-05 17:38:32 +02:00
Michael Eischer
d6309961c5
deduplicate the snapshot filter cli option setup
2022-09-04 10:27:33 +02:00
Michael Eischer
8b4dd70013
migrate: Report why an migration cannot be applied
...
Just returning that `Migration upgrade cannot be applied: check failed`
is not too useful when running `migrate upgrade_repo_v2`.
2022-09-03 11:49:31 +02:00
Michael Eischer
7689d6c679
normalize help text for host, tag and path options
2022-09-03 00:06:38 +02:00
Michael Eischer
6c69f08a7b
Merge pull request #3905 from DRON-666/haspaths-linear
...
Reduce quadratic time complexity of `Snapshot.HasPaths`
2022-08-30 20:35:56 +02:00
Michael Eischer
3e70bac56e
Merge pull request #3898 from MichaelEischer/fix-copy-hang
...
don't hang when `copy` uses a single connection
2022-08-30 20:23:39 +02:00
DRON-666
2a630c51c1
Add changelog
2022-08-30 20:22:07 +02:00
DRON-666
d0f1060df7
Fix quadratic time complexity of Snapshot.HasPaths
2022-08-30 04:38:17 +03:00
Michael Eischer
f481ad64c8
Merge pull request #3904 from lbausch/add-newline
...
Add newline to keep prompt intact
2022-08-29 21:43:18 +02:00
Lorenz Bausch
7ddd803e46
Add newline to keep prompt intact
2022-08-29 17:37:49 +02:00
Michael Eischer
e5b2c4d571
b2: sniff the error that caused init retry loops
2022-08-28 17:46:03 +02:00
Michael Eischer
dc2db2de5e
b2: cancel connection setup after a minute
...
If the connection to B2 fails, the library enters an endless loop.
2022-08-28 14:56:17 +02:00
Michael Eischer
7682149c9d
repository: cleanup copy connection count check
2022-08-28 11:40:56 +02:00
Michael Eischer
b03277ead5
repository: don't hang when copying using a single connection
2022-08-28 11:40:31 +02:00
Michael Eischer
1b233c4e2e
Merge pull request #2661 from creativeprojects/issue-1734
...
"Everything is ok" message after retrying
2022-08-28 11:04:59 +02:00
Fred
4042db5169
Add changelog
2022-08-27 22:36:19 +02:00
Fred
be6baaec12
Add success callback to the backend
2022-08-27 22:27:15 +02:00
Fred
baf58fbaa8
Add unit tests
2022-08-27 22:21:06 +02:00
Fred
d629333efe
Add function to notify of success after retrying
2022-08-27 22:21:06 +02:00
Alexander Neumann
c169e37139
Merge pull request #3895 from MichaelEischer/refactor-cat-key
...
cat: Simplify implementation of 'cat key'
2022-08-27 18:40:46 +02:00
Michael Eischer
1b4af0c6e5
cat: Simplify implementation of 'cat key'
2022-08-26 23:21:51 +02:00
Michael Eischer
3174641ca4
add changelog for mount exit code filtering
2022-08-26 23:17:04 +02:00
Michael Eischer
5478ab22c5
mount: return exit code 0 after receiving a SIGINT
2022-08-26 23:07:07 +02:00
Michael Eischer
d768c1c3e4
Allow cleanup handlers to filter the exit code
2022-08-26 23:04:59 +02:00
Michael Eischer
908f7441fe
Merge pull request #3885 from MichaelEischer/delete-fixes
...
Improve reliability of upload retries and B2 file deletions
2022-08-26 22:30:50 +02:00
Michael Eischer
4c90d91d4d
backend: Test that failed uploads are not removed for backends with atomic replace
2022-08-26 21:20:52 +02:00
Michael Eischer
694dfa026a
add changelog for reliable B2 deletes
2022-08-26 21:20:46 +02:00
MichaelEischer
582167d671
Merge pull request #3882 from MichaelEischer/sftp-init-single-connection
...
sftp: Only connect once to server during `init`
2022-08-26 21:13:28 +02:00
MichaelEischer
3822ded0b3
Merge pull request #3877 from MichaelEischer/no-env-in-help
...
Do not include the actual values of environment variables in help output
2022-08-26 20:59:54 +02:00
Michael Eischer
cf0a8d7758
sftp: Only connect once for repository creation
...
This is especially useful if ssh asks for a password or if closing the
initial connection could return an error due to a problematic server
implementation.
2022-08-26 20:50:40 +02:00
Michael Eischer
dd7cd5b9b3
fuse: remove unused context parameter
2022-08-26 20:48:48 +02:00
Michael Eischer
a0c1ae9f90
mount: Correctly return context.Canceled for interrupted syscalls
...
bazil/fuse expects us to return context.Canceled to signal that a
syscall was successfully interrupted. Returning a wrapped version of
that error however causes the fuse library to signal an EIO (input/output
error). Thus unwrap context.Canceled errors before returning them.
2022-08-26 20:48:48 +02:00
Michael Eischer
5d0649faaf
Update help output in docs
2022-08-26 20:44:01 +02:00
Michael Eischer
faa4597af1
Set name for option values of cli
2022-08-26 20:42:34 +02:00
Michael Eischer
6ed157aee6
Do not include the actual values of environment variables in help output
...
This results in printing a `(default: $ENV) (default: value)` suffix for
the corresponding options which looks strange. In addition, some of the
environment variables might contain secrets which should not be
displayed.
2022-08-26 20:39:54 +02:00
MichaelEischer
f7808245aa
Merge pull request #3878 from MichaelEischer/cheaper-cache-load
...
cache: Just try to open cache entry without calling stat first
2022-08-26 20:33:36 +02:00
MichaelEischer
bee15dd555
Merge pull request #3879 from MichaelEischer/mem-optimize
...
Some random (minor) memory-allocation optimizations
2022-08-26 20:33:02 +02:00
MichaelEischer
0e1d082b12
Merge pull request #3886 from MichaelEischer/recommend-s3-over-b2
...
doc: recommend usage of B2's S3 API
2022-08-26 20:29:05 +02:00
Alexander Neumann
d464543171
Update repo version table
2022-08-25 21:30:25 +02:00
Alexander Neumann
6b40456db7
Set development version for 0.14.0
2022-08-25 19:55:05 +02:00
Alexander Neumann
1bc87e1718
Add version for 0.14.0
2022-08-25 19:55:00 +02:00
Alexander Neumann
c1a5da56e3
Update manpages and auto-completion
2022-08-25 19:55:00 +02:00
Alexander Neumann
193c62dfc3
Generate CHANGELOG.md for 0.14.0
2022-08-25 19:54:02 +02:00
Alexander Neumann
a825e0d409
Prepare changelog for 0.14.0
2022-08-25 19:54:01 +02:00
rawtaz
b824d8cdcc
Merge pull request #3891 from MichaelEischer/fix-secondary-repo-typo
...
Fix typo in the environment variable name for `--from-password-file`
2022-08-24 22:38:47 +02:00
Michael Eischer
a2e89234fc
Fix typo in the environment variable name for --from-password-file
2022-08-24 22:25:18 +02:00
MichaelEischer
b4ae05627f
Merge pull request #3876 from MichaelEischer/document-aws-session-token
...
doc: document aws session token
2022-08-24 22:20:26 +02:00
MichaelEischer
bd7bca2b51
Merge pull request #3889 from restic/prepare-0-14
...
Polish changelog entries
2022-08-24 21:28:44 +02:00
MichaelEischer
be90a565cc
Merge pull request #3887 from MichaelEischer/rclone-permanent-error
...
rclone: Return a permanent error if rclone already exited
2022-08-24 21:19:00 +02:00
Michael Eischer
7a5d29ce24
Further changelog polishing
2022-08-24 21:13:14 +02:00
MichaelEischer
d198a77d86
Merge pull request #3888 from restic/doc-prepare-repo
...
doc: Improve/clarify preparing and versions of repositories
2022-08-24 20:40:49 +02:00
Leo R. Lundgren
cb5a61c46e
doc: Improve/clarify preparing and versions of repositories
2022-08-24 02:08:46 +02:00
Leo R. Lundgren
ee6b9dc492
Polish changelog entries
2022-08-24 01:23:51 +02:00
Michael Eischer
506d92e87c
rclone: Return a permanent error if rclone already exited
...
rclone can exit early for example when the connection to rclone is
relayed for example via ssh: `-o rclone.program='ssh user@example.org
forced-command'`
2022-08-23 22:05:04 +02:00
MichaelEischer
ad6eabbfa5
Merge pull request #3883 from MichaelEischer/update-dependencies
...
Update dependencies
2022-08-23 21:44:38 +02:00
Michael Eischer
7681a63fdb
restic: Cleanup xattr error handling for Solaris
...
Since xattr 0.4.8 (https://github.com/pkg/xattr/pull/68 ) returns ENOTSUP
similar to Linux.
2022-08-23 21:25:15 +02:00
Michael Eischer
99e4ccbd94
remain compatible with go 1.15
2022-08-23 21:25:14 +02:00
Michael Eischer
22f46c18f9
downgrade bazil/fuse again to retain macOS support
2022-08-23 21:24:47 +02:00
Michael Eischer
6db979b3a6
update dependencies
2022-08-23 21:24:46 +02:00
MichaelEischer
9cdc8da10f
Merge pull request #3881 from MichaelEischer/cleanup-release-helper
...
Cleanup release helper
2022-08-23 21:10:57 +02:00
MichaelEischer
b51e73e78f
Merge pull request #3884 from MichaelEischer/fix-index-saving-progress
...
repository: Do not report ignored packs in EachByPack
2022-08-23 21:10:07 +02:00
MichaelEischer
98dcd0a887
Merge pull request #3874 from MichaelEischer/disk-wear-note
...
Add note that larger packs increase disk wear
2022-08-21 22:18:08 +02:00
Michael Eischer
e0d6bf525c
doc: fix typo
2022-08-21 19:12:19 +02:00
Michael Eischer
5d0523e2f1
Add note that larger packs increase disk wear
2022-08-21 19:12:19 +02:00
Michael Eischer
c586a5e20f
doc: recommend usage of B2's S3 API
2022-08-21 11:39:03 +02:00
Michael Eischer
623556bab6
b2: Increase list size to maximum
...
Just request as many files as possible in one call to reduce the number
of network roundtrips.
2022-08-21 11:20:03 +02:00
Michael Eischer
de0162ea76
backend/retry: Overwrite failed uploads instead of deleting them
...
For backends which are able to atomically replace files, we just can
overwrite the old copy, if it is necessary to retry an upload. This has
the benefit of issuing one operation less and might be beneficial if a
backend storage, due to bugs or similar, could mix up the order of the
upload and delete calls.
2022-08-21 11:14:53 +02:00
Michael Eischer
fc506f8538
b2: Repeat deleting until all file versions are removed
...
When hard deleting the latest file version on B2, this uncovers earlier
versions. If an upload required retries, multiple version might exist
for a file. Thus to reliably delete a file, we have to remove all
versions of it.
2022-08-21 11:11:00 +02:00
Michael Eischer
cc4728d287
repository: Do not report ignored packs in EachByPack
...
Ignored packs were reported as an empty pack by EachByPack. The most
immediate effect of this is that the progress bar for rebuilding the
index reports processing more packs than actually exist.
2022-08-21 10:38:40 +02:00
Michael Eischer
c3374b3ea5
helper: download modules as first step
...
There's no use in running that step in parallel.
2022-08-20 12:11:54 +02:00
Michael Eischer
7f0929e519
helper: Reduce number of parallel builds a bit
...
The go compiler is already parallelized. The high concurrency caused my
podman container to hit a resource limit.
2022-08-20 12:10:48 +02:00
Michael Eischer
ed94678820
helper: cleanups
2022-08-20 12:10:29 +02:00
Michael Eischer
e530d422a0
helper: don't setup cmd paths twice
2022-08-20 12:09:42 +02:00
Michael Eischer
7a992fc794
repository: Reduce buffer reallocations in ForAllIndexes
...
Previously the buffer was grown incrementally inside `repo.LoadUnpacked`.
But we can do better as we already know how large the index will be.
Allocate a bit more memory to increase the chance that the buffer can be
reused in the future.
2022-08-19 21:13:40 +02:00
Michael Eischer
77b1980d8e
repository: MasterIndex.Packs: reduce allocations
2022-08-19 21:10:43 +02:00
Michael Eischer
6ff9517e45
repository: MasterIndex.ListPacks / Index.EachByPack allow earlier GC
...
Allow earlier garbage collection of some of the intermediate data
structures.
2022-08-19 21:06:33 +02:00
Michael Eischer
ce902aac67
cache: Just try to open cache entry without calling stat first
...
Instead of first checking whether a file is in the repository cache and
then opening it, we just can open the file. This saves one stat call. If
the file is in the cache, everything is fine and otherwise the code
follows its normal fallback path.
2022-08-19 20:59:06 +02:00
Michael Eischer
b6c86ababe
doc: document aws session token
2022-08-19 20:41:15 +02:00
MichaelEischer
0d9ac78437
Merge pull request #3873 from MichaelEischer/gofmt-comments
...
gofmt comments
2022-08-19 19:54:30 +02:00
MichaelEischer
7e96a5af62
Merge pull request #3872 from MichaelEischer/fuse-fix
...
mount: Only remember successful snapshot refreshes
2022-08-19 19:21:29 +02:00
Michael Eischer
f414db987d
gofmt all files
...
Apparently the rules for comment formatting have changed with go 1.19.
2022-08-19 19:12:26 +02:00
Michael Eischer
522406b4f0
mount: Only remember successful snapshot refreshes
...
If the context provided by the fuse library is canceled before the index
was loaded this could lead to missing snapshots.
2022-08-19 19:07:07 +02:00
MichaelEischer
dbca93da28
Merge pull request #3742 from MichaelEischer/from-repo
...
copy: replace --repo2 with --from-repo
2022-08-19 19:01:04 +02:00
MichaelEischer
b4dfab002a
Merge pull request #3691 from greatroar/tag-filenames
...
Sanitize tags when used as filenames by restic mount
2022-08-19 18:33:44 +02:00
Michael Eischer
2758d76b77
copy: replace --repo2 with --from-repo
...
`init` and `copy` use `--repo2` with two different meaning which has
proven to be confusing for users. `--from-repo` now consistently marks a
source repository from which data is read. `--repo` is now always the
target/destination repository.
2022-08-19 18:33:26 +02:00
Michael Eischer
af50fe9ac0
mount: Map slashes in tags to underscores
...
Suggested-by: greatroar <>
2022-08-19 18:17:57 +02:00
MichaelEischer
4cccffab58
Merge pull request #3862 from restic/3861-forget-invalid-policy
...
forget: Error when invalid unit is given in duration policy
2022-08-18 20:45:26 +02:00
Michael Eischer
2ea6c82cf6
comment cleanup
...
gofmt reformatted the comment
2022-08-18 20:15:38 +02:00
Michael Eischer
bb27f7408c
forget: Fail test if duration parsing error is missing
2022-08-18 20:14:09 +02:00
MichaelEischer
c4b3a154ba
Merge pull request #3865 from restic/rawtaz-doc-comp-typo
...
doc: Fix typo in compression section
2022-08-18 19:37:44 +02:00
rawtaz
2cb2aa31cd
doc: Fix typo in compression section
2022-08-12 22:17:38 +02:00
Leo R. Lundgren
6f517858e8
forget: Error when invalid unit is given in duration policy
2022-08-10 13:37:26 +02:00
rawtaz
f0bb4f8708
Merge pull request #3857 from restic/rawtaz-gcs-account
...
doc: Update links to GCS documentation
2022-08-08 22:52:30 +02:00
Michael Eischer
40c8755b13
doc: Update more links to GCS documentation
2022-08-08 20:00:44 +02:00
rawtaz
f673068dbb
doc: Update link to GCS documentation
...
Updates the link to Google Cloud Storage documentation about creating a service account key.
2022-08-08 12:51:43 +02:00
rawtaz
f26231c9e6
Merge pull request #3852 from MichaelEischer/snapshots-processed
...
stats: Add snapshots count to json output
2022-08-08 00:05:15 +02:00
MichaelEischer
80e3efffef
Merge pull request #3855 from mattxtaz/master
...
Fix typo with double percentage in help text
2022-08-07 22:44:36 +02:00
mattxtaz
01ab36336f
Fix typo with double percentage in help text
2022-08-07 20:21:05 +01:00
Michael Eischer
6e92d852a8
stats: Add snapshots count to json output
2022-08-07 15:44:09 +02:00
MichaelEischer
9ad3ad5972
Merge pull request #3850 from lbausch/go1.19
...
Update tests to Go 1.19
2022-08-07 14:56:17 +02:00
MichaelEischer
2930a102de
Merge pull request #3731 from metalsp0rk/feature/min-packsize-flag
...
Feature: min packsize flag
2022-08-07 14:54:45 +02:00
MichaelEischer
8fa64a8f99
Merge pull request #3036 from greatroar/refactor-fuse
...
Clean up internal/fuse
2022-08-07 14:46:30 +02:00
Michael Eischer
f3fdc66b32
restic: Use stable sorting in snapshot policy
...
sort.Sort is not guaranteed to be stable. Go 1.19 has changed the
sorting algorithm which resulted in changes of the sort order. When
comparing snapshots with identical timestamp but different paths and
tags lists, there is not meaningful order among them. So just keep their
order stable.
2022-08-07 14:10:40 +02:00
Lorenz Bausch
b82f4824f0
Bump golangci-lint version
2022-08-07 14:10:39 +02:00
Lorenz Bausch
0b9b4c52ad
Update tests to Go 1.19
2022-08-07 14:10:39 +02:00
Michael Eischer
0b7291b8b2
mount: Fix parent inode used by snapshots dir
2022-08-07 13:03:32 +02:00
greatroar
cfa80e2c6b
mount: remove unused inode field from root node
2022-08-07 13:03:26 +02:00
MichaelEischer
74ae76036f
Merge pull request #2913 from aawsome/mount-snapshot-slashes
...
mount: Make snapshots dir structure customizable
2022-08-07 12:27:59 +02:00
MichaelEischer
09497aec02
Merge pull request #3826 from MichaelEischer/debug-log-for-release
...
Debug log for release build
2022-08-07 12:15:27 +02:00
Michael Eischer
83b4c50ee3
Mention --snapshot-template and --time-template in changelog
2022-08-07 12:13:06 +02:00
Michael Eischer
caa17988a3
fuse: Redesign snapshot dirstruct
...
Cleanly separate the directory presentation and the snapshot directory
structure. SnapshotsDir now translates the dirStruct into a format
usable by the fuse library and contains only minimal special case rules.
All decisions have moved into SnapshotsDirStructure which now creates a
fully preassembled tree data structure.
2022-08-07 12:13:06 +02:00
Michael Eischer
a3e48da3a3
Add changelog for DEBUG_LOG available in release builds
2022-08-05 23:49:39 +02:00
Michael Eischer
b3cdee66a9
update documentation to reflect DEBUG_LOG for release builds
2022-08-05 23:49:39 +02:00
Michael Eischer
1ed775e3a8
debug: support roundtripper logging also for release builds
...
Different from debug builds do not use the eofDetectRoundTripper if
logging is disabled.
2022-08-05 23:49:39 +02:00
Michael Eischer
38becfc436
debug: enable debug support for release builds
2022-08-05 23:49:39 +02:00
Michael Eischer
82c268c917
Remove unused hooks mechanism
2022-08-05 23:49:39 +02:00
Michael Eischer
7266f07c87
repository: StreamPack in parts if there are too large gaps
...
For large pack sizes we might be only interested in the first and last
blob of a pack file. Thus stream a pack file in multiple parts if the
gaps between requested blobs grow too large.
2022-08-05 23:48:36 +02:00
Michael Eischer
55a11c1396
Reword prune --repack-small description
2022-08-05 23:48:36 +02:00
Michael Eischer
eaf43607f9
Add note that pack-size is not an exact limit
2022-08-05 23:48:36 +02:00
Michael Eischer
7f3b2be1e8
s3: Disable multipart uploads below 200MB
2022-08-05 23:48:36 +02:00
Michael Eischer
176b387d98
Always repack very small pack files
2022-08-05 23:48:36 +02:00
Michael Eischer
324935cb80
Only repack small files if there are multiple of them
2022-08-05 23:48:34 +02:00
Michael Eischer
1b076cda97
rename option to --pack-size
2022-08-05 23:47:43 +02:00
Michael Eischer
d7e2892048
Add changelog for packsize option
2022-08-05 23:47:43 +02:00
Michael Eischer
8a44258b6f
update restic help snippets in documentation
2022-08-05 23:47:43 +02:00
Michael Eischer
420ddc03c9
rework pack size parameter documentation
2022-08-05 23:47:43 +02:00
Kyle Brennan
e43be84eb8
document minPackSize
2022-08-05 23:47:41 +02:00
Kyle Brennan
1e3f05c3f1
repository: prevent header overfill
2022-08-05 23:47:12 +02:00
Michael Eischer
6a6d313c9a
prune: reduce priority of repacking small packs
2022-08-05 23:47:12 +02:00
Kyle Brennan
0269381b8d
prune: add repack-small parameter
2022-08-05 23:47:12 +02:00
Michael Eischer
0a6fa602c8
add option for setting min pack size
2022-08-05 23:47:12 +02:00
Michael Eischer
2db7733ee3
fuse: remove unused MetaDir
2022-08-05 23:46:46 +02:00
Michael Eischer
f678f7cb04
fuse: cleanup test
2022-08-05 23:46:46 +02:00
Alexander Weiss
1751afae26
Make snapshots dirs in mount command customizable
2022-08-05 23:46:46 +02:00
Alexander Weiss
57f4003f2f
Generalize fuse snapshot dirs implemetation
...
+ allow "/" in tags and snapshot template
2022-08-05 23:46:46 +02:00
Alexander Weiss
696c18e031
Add possibility to set snapshot ID (used in test)
2022-08-05 23:46:46 +02:00
MichaelEischer
04a8ee80fb
Merge pull request #3829 from MichaelEischer/prune-refactor
...
Split prune into slightly small functions
2022-08-05 23:29:52 +02:00
MichaelEischer
15679be858
Merge pull request #3841 from MichaelEischer/compression-env
...
Add environment variable RESTIC_COMPRESSION
2022-08-05 23:20:04 +02:00
MichaelEischer
7d14b1baf1
Merge pull request #3845 from greatroar/solaris-xattr
...
internal/restic: Handle EINVAL for xattr on Solaris
2022-08-04 22:04:52 +02:00
greatroar
ad6ac680af
internal/restic: Handle EINVAL for xattr on Solaris
...
Also make the errors a bit less verbose by not prepending the operation,
since pkg/xattr already does that. Old errors looked like
Listxattr: xattr.list /myfiles/.zfs/snapshot: invalid argument
2022-08-01 12:45:17 +02:00
MichaelEischer
846d021db5
Merge pull request #3840 from greatroar/sftp-init
...
Speed up restic init over slow SFTP links
2022-07-31 19:54:59 +02:00
greatroar
b9fa6e05bd
Add changelog for #3837/#3840
2022-07-30 22:53:52 +02:00
Michael Eischer
73053674d9
repository: Test fallback to existing blobs
2022-07-30 17:37:07 +02:00
Michael Eischer
e85a21eda2
prune: move code
2022-07-30 17:37:07 +02:00
Michael Eischer
623770eebb
repository: try to recover from invalid blob while repacking
...
If a blob that should be kept is invalid, Repack will now try to request
the blob using LoadBlob. Only return an error if that fails.
2022-07-30 17:37:07 +02:00
Michael Eischer
d0590b7841
prune: Add internal integrity check
...
After repacking every blob that should be kept must have been repacked.
We have seen a few cases in which a single blob went missing, which
could have been caused by a bitflip somewhere. This sanity check might
help catch some of these cases.
2022-07-30 17:37:07 +02:00
Michael Eischer
5cbde03eae
prune: split into smaller functions
2022-07-30 17:37:07 +02:00
Alexander Weiss
7643237da5
prune: separate collecting/printing/pruning
2022-07-30 17:37:07 +02:00
Michael Eischer
5723c72eb1
Add environment variable RESTIC_COMPRESSION
2022-07-30 16:21:53 +02:00
greatroar
2bdc40e612
Speed up restic init over slow SFTP links
...
pkg/sftp.Client.MkdirAll(d) does a Stat to determine if d exists and is
a directory, then a recursive call to create the parent, so the calls
for data/?? each take three round trips. Doing a Mkdir first should
eliminate two round trips for 255/256 data directories as well as all
but one of the top-level directories.
Also, we can do all of the calls concurrently. This may reintroduce some
of the Stat calls when multiple goroutines try to create the same
parent, but at the default number of connections, that should not be
much of a problem.
2022-07-30 13:09:08 +02:00
greatroar
23ebec717c
Remove stale comments from backend/sftp
...
The preExec and postExec functions were removed in
0bdb131521
from 2018.
2022-07-30 13:07:25 +02:00
MichaelEischer
4ffd479ba4
Merge pull request #3773 from MichaelEischer/efficient-dir-json
...
Reduce memory usage for large directories/files
2022-07-23 17:47:32 +02:00
Michael Eischer
2ba14160de
Add changelog for the optimized tree serialization
2022-07-23 14:49:08 +02:00
Michael Eischer
4a10ebed15
archiver: reduce memory usage for large files
...
FutureBlob now uses a Take() method as a more memory-efficient way to
retrieve the futures result. In addition, futures are now collected
while saving the file. As only a limited number of blobs can be queued
for uploading, for a large file nearly all FutureBlobs already have
their result ready, such that the FutureBlob object just consumes
memory.
2022-07-23 14:45:07 +02:00
Michael Eischer
b817681a11
archiver: Incrementally serialize tree nodes
...
That way it is not necessary to keep both the Nodes forming a Tree and
the serialized JSON version in memory.
2022-07-23 14:45:07 +02:00
Michael Eischer
c206a101a3
archiver: unify FutureTree/File into futureNode
...
There is no real difference between the FutureTree and FutureFile
structs. However, differentiating both increases the size of the
FutureNode struct.
The FutureNode struct is now only 16 bytes large on 64bit platforms.
That way is has a very low overhead if the corresponding file/directory
was not processed yet.
There is a special case for nodes that were reused from the parent
snapshot, as a go channel seems to have 96 bytes overhead which would
result in a memory usage regression.
2022-07-23 14:45:07 +02:00
Michael Eischer
32f4997733
archiver: remove unused fileInfo from progress callback
2022-07-23 14:16:23 +02:00
Michael Eischer
dcb00fd2d1
archiver: cleanup Saver interface
2022-07-23 14:16:23 +02:00
Michael Eischer
79321a195c
archiver: remove dead attribute from FutureNode
2022-07-23 14:16:23 +02:00
MichaelEischer
049f4c4144
Merge pull request #3730 from MichaelEischer/stricter-check
...
Let `check` warn about legacy variants of the repo format version 1
2022-07-23 14:14:50 +02:00
Michael Eischer
3bf53da672
Add changelog for stricter checks
2022-07-23 11:21:26 +02:00
Michael Eischer
5a6f2f9fa0
Fix S3 legacy layout migration
2022-07-23 11:19:32 +02:00
Michael Eischer
04e49924fb
checker: Fix S3 legacy layout detection
2022-07-23 11:19:32 +02:00
Michael Eischer
768c890fcb
check: Deprecate `--check-unused`
...
Unused blobs are not a problem but rather expected to exist now that
prune by default does not remove every unused blob. However, the option
has caused questions from users whether a repository is damaged or not,
so just remove that option.
Note that the remaining code is left intact as it is still useful for
our test cases.
2022-07-23 11:19:32 +02:00
Michael Eischer
fcb3ddf181
check: Complain about usage of s3 legacy layout
2022-07-23 11:19:32 +02:00
Michael Eischer
8b8bd4e8ac
check: complain about mixed pack files
2022-07-23 11:19:32 +02:00
MichaelEischer
443cc49afd
Merge pull request #3830 from MichaelEischer/cleanup-repo
...
Extract Load/SaveTree/JSONUnpacked from repository
2022-07-23 10:46:13 +02:00
MichaelEischer
1f5369e072
Merge pull request #3831 from MichaelEischer/move-code
...
Move code out of the restic package and consolidate backend specific code
2022-07-23 10:33:05 +02:00
MichaelEischer
827ab02eea
Merge pull request #3661 from rgammans/azure_sas_support
...
add SAS authentication option for Azure repos
2022-07-23 10:32:03 +02:00
MichaelEischer
e9c39442fb
Merge pull request #3827 from MichaelEischer/backup-doc
...
backup: clarify usage string
2022-07-23 10:31:33 +02:00
Michael Eischer
9729e6d7ef
backend: extract readerat from restic package
2022-07-17 15:29:09 +02:00
Michael Eischer
c44b21d366
restorer: extract hardlinks index from restic package
2022-07-17 13:45:42 +02:00
Michael Eischer
8c11fc3ec9
crypto: move crypto buffer helpers
2022-07-17 13:42:23 +02:00
Michael Eischer
a0cef9f247
limiter: move to internal/backend
2022-07-17 13:40:15 +02:00
Michael Eischer
163ab9c025
mock: move to internal/backend
2022-07-17 13:40:06 +02:00
Michael Eischer
89d3ce852b
repository: extract Load/StoreJSONUnpacked
...
A Load/Store method for each data type is much clearer. As a result the
repository no longer needs a method to load / store json.
2022-07-17 13:22:00 +02:00
Michael Eischer
fbcbd5318c
repository: extract LoadTree/SaveTree
...
The repository has no real idea what a Tree is. So these methods never
belonged there.
2022-07-17 13:11:28 +02:00
MichaelEischer
d9ea1e9ee2
Merge pull request #3290 from aawsome/prune-handle-duplicates
...
prune: Handle duplicate blobs more efficiently
2022-07-17 11:51:54 +02:00
Michael Eischer
715d457aad
prune: code cleanups
2022-07-17 11:41:56 +02:00
Michael Eischer
9be1bd2acc
prune: handle very high duplication of some blobs
...
Suggested-By: Alexander Weiss <alex@weissfam.de>
2022-07-17 11:39:56 +02:00
Alexander Weiss
7478cbf70e
prune: Enhance treatment of duplicates
2022-07-17 00:22:23 +02:00
Michael Eischer
b2043e8198
backup: clarify usage string
...
Using the `--files-from` options it is possible to run `backup` without
specifying any source paths directly on the command line.
2022-07-17 00:03:22 +02:00
Michael Eischer
5639c41b6a
azure: Strip ? prefix from sas token
2022-07-16 23:55:18 +02:00
Roger Gammans
64a7ec5341
azure: add SAS authentication option
2022-07-16 23:55:18 +02:00
MichaelEischer
6cbeb4a9f9
Merge pull request #3825 from restic/rawtaz-doc-gdrive
...
doc: Add note about using rclone for Google Drive
2022-07-16 19:21:59 +02:00
rawtaz
f5c219f5a2
doc: Add note about using rclone for Google Drive
...
It wasn't clear that Google Cloud Storage and Google Drive are two different services and that one should use the rclone backend for the latter. This commit adds a note with this information.
2022-07-16 13:22:38 +02:00
MichaelEischer
d71b29221b
Merge pull request #3822 from JsBergbau/doc-max-compression
...
Added hint for --compression max in migration process
2022-07-12 23:02:45 +02:00
Michael Eischer
71ff6b77f0
doc: Rework hint to repack with max compression
2022-07-12 21:24:40 +02:00
MichaelEischer
6970d05d47
Merge pull request #3741 from lbausch/repository-wording
...
Wording: change repo to repository
2022-07-12 21:15:33 +02:00
MichaelEischer
3934480da4
Merge pull request #3819 from lbausch/restore-validate-patterns
...
restore: validate include/exclude patterns
2022-07-12 20:58:41 +02:00
MichaelEischer
71a0157c2c
Merge pull request #3820 from lbausch/patch-1
...
Fix wording in changelog template
2022-07-12 20:51:52 +02:00
Michael Eischer
2aad6f24b5
doc: update sample help output
2022-07-12 20:49:01 +02:00
Michael Eischer
ec4dfa3c66
Wording: replace further repo occurrences with repository
2022-07-12 20:48:01 +02:00
JsBergbau
8d3f04aefa
Added hint for --compression max in migration process
...
Added hint for --compression max in migration process. Since this is a onetime process users should be aware of this and consider this step.
2022-07-12 00:08:26 +02:00
Lorenz Bausch
b609523582
Add changelog entry
2022-07-09 22:24:56 +02:00
lbausch
ac96a4138d
Fix wording in changelog template
2022-07-09 22:13:54 +02:00
Lorenz Bausch
7e36ec279d
Test restore fails when using invalid patterns
2022-07-08 20:09:26 +02:00
Lorenz Bausch
be524f0b78
Add testRunRestoreAssumeFailure function
2022-07-08 20:09:25 +02:00
Lorenz Bausch
9a7db6675c
Restore: validate provided patterns
2022-07-08 20:09:25 +02:00
Lorenz Bausch
d6e3c7f28e
Wording: change repo to repository
2022-07-08 20:05:35 +02:00
MichaelEischer
98a3125ce4
Merge pull request #3816 from mattxtaz/master
...
Fix minor typo in docs
2022-07-05 21:30:26 +02:00
mattxtaz
87d899c099
Fix minor typo in docs
2022-07-05 20:06:27 +01:00
Jerome Küttner
6f3883c9d2
add changelog
2022-07-05 08:47:48 +02:00
Alexander Neumann
545220803b
Merge pull request #3813 from MichaelEischer/fix-blob-saver-data-race
...
Fix data race in blob_saver
2022-07-04 08:10:01 +02:00
Michael Eischer
ce89018902
Fix data race in blob_saver
...
After the `BlobSaver` job is submitted, the buffer can be released and
reused by another `FileSaver` even before `BlobSaver.Save` returns. That
FileSaver will the change `buf.Data` leading to wrong backup statistics.
Found by `go test -race ./...`:
WARNING: DATA RACE
Write at 0x00c0000784a0 by goroutine 41:
github.com/restic/restic/internal/archiver.(*FileSaver).saveFile()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:176 +0x789
github.com/restic/restic/internal/archiver.(*FileSaver).worker()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:242 +0x2af
github.com/restic/restic/internal/archiver.NewFileSaver.func2()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:88 +0x5d
golang.org/x/sync/errgroup.(*Group).Go.func1()
/home/michael/go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x91
Previous read at 0x00c0000784a0 by goroutine 29:
github.com/restic/restic/internal/archiver.(*BlobSaver).Save()
/home/michael/Projekte/restic/restic/internal/archiver/blob_saver.go:57 +0x1dd
github.com/restic/restic/internal/archiver.(*BlobSaver).Save-fm()
<autogenerated>:1 +0xac
github.com/restic/restic/internal/archiver.(*FileSaver).saveFile()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:191 +0x855
github.com/restic/restic/internal/archiver.(*FileSaver).worker()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:242 +0x2af
github.com/restic/restic/internal/archiver.NewFileSaver.func2()
/home/michael/Projekte/restic/restic/internal/archiver/file_saver.go:88 +0x5d
golang.org/x/sync/errgroup.(*Group).Go.func1()
/home/michael/go/pkg/mod/golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c/errgroup/errgroup.go:57 +0x91
2022-07-03 14:47:53 +02:00
MichaelEischer
b6a38d43b3
Merge pull request #3611 from MichaelEischer/auto-concurrency
...
Adjust worker goroutines to number of backend connections
2022-07-03 12:33:32 +02:00
Michael Eischer
3af9c2cc58
Document automatic CPU/IO-concurrency
2022-07-03 12:19:26 +02:00
Michael Eischer
6f53ecc1ae
adapt workers based on whether an operation is CPU or IO-bound
...
Use runtime.GOMAXPROCS(0) as worker count for CPU-bound tasks,
repo.Connections() for IO-bound task and a combination if a task can be
both. Streaming packs is treated as IO-bound as adding more worker
cannot provide a speedup.
Typical IO-bound tasks are download / uploading / deleting files.
Decoding / Encoding / Verifying are usually CPU-bound. Several tasks are
a combination of both, e.g. for combined download and decode functions.
In the latter case add both limits together. As the backends have their
own concurrency limits restic still won't download more than
repo.Connections() files in parallel, but the additional workers can
decode already downloaded data in parallel.
2022-07-03 12:19:26 +02:00
MichaelEischer
cd50feb66f
Merge pull request #3489 from MichaelEischer/async-pack-uploads
...
Asynchronously upload pack files
2022-07-03 11:56:05 +02:00
Michael Eischer
74df9d5998
Add changelog for async pack uploads
2022-07-03 11:34:01 +02:00
Michael Eischer
dbb5860dc9
Document connections and compression option
2022-07-03 11:19:24 +02:00
Michael Eischer
753e56ee29
repository: Limit to a single pending pack file
...
Use only a single not completed pack file to keep the number of open and
active pack files low. The main change here is to defer hashing the pack
file to the upload step. This prevents the pack assembly step to become
a bottleneck as the only task is now to write data to the temporary pack
file.
The tests are cleaned up to no longer reimplement packer manager
functions.
2022-07-02 22:42:34 +02:00
Michael Eischer
fa25d6118e
archiver: Reduce tree saver concurrency
...
Large amount of tree savers have no obvious benefit, however they can
increase the amount of (potentially large) trees kept in memory.
2022-07-02 22:42:34 +02:00
Michael Eischer
bba1e81719
archiver: Limit blob saver count to GOMAXPROCS
...
Now with the asynchronous uploaders there's no more benefit from using
more blob savers than we have CPUs. Thus use just one blob saver for
each CPU we are allowed to use.
2022-07-02 22:42:34 +02:00
Michael Eischer
120ccc8754
repository: Rework blob saving to use an async pack uploader
...
Previously, SaveAndEncrypt would assemble blobs into packs and either
return immediately if the pack is not yet full or upload the pack file
otherwise. The upload will block the current goroutine until it
finishes.
Now, the upload is done using separate goroutines. This requires changes
to the error handling. As uploads are no longer tied to a SaveAndEncrypt
call, failed uploads are signaled using an errgroup.
To count the uploaded amount of data, the pack header overhead is no
longer returned by `packer.Finalize` but rather by
`packer.HeaderOverhead`. This helper method is necessary to continue
returning the pack header overhead directly to the responsible call to
`repository.SaveBlob`. Without the method this would not be possible,
as packs are finalized asynchronously.
2022-07-02 22:42:34 +02:00
Michael Eischer
bba4c69a2a
tag: Remove unnecessary flush call
2022-07-02 22:42:23 +02:00
MichaelEischer
3e1de52e0a
Merge pull request #3805 from greatroar/global
...
cmd/restic, limiter: Move config knowledge to internal packages
2022-07-02 21:56:35 +02:00
MichaelEischer
621023a50b
Merge pull request #3772 from MichaelEischer/fix-mixed-index
...
rebuild-index: correctly rebuild index for mixed packs
2022-07-02 20:10:02 +02:00
MichaelEischer
90e9c5c4cc
Merge pull request #3729 from MichaelEischer/full-ids-in-check
...
Include full IDs in check output
2022-07-02 20:09:39 +02:00
MichaelEischer
7137034517
Merge pull request #3811 from restic/fix-secret-string-crash
...
Don't crash if SecretString is uninitialized
2022-07-02 19:56:36 +02:00
Michael Eischer
cdaf9b4f26
Don't crash if SecretString is uninitialized
2022-07-02 19:44:28 +02:00
Michael Eischer
5e0f1c3cef
check: remove dead code
2022-07-02 19:28:57 +02:00
Michael Eischer
0df022fa6d
check: Print full ids
...
The short ids are not always unique. In addition, recovering from
damages is easier when having the full ids as that makes it easier to
access the corresponding files.
2022-07-02 19:28:57 +02:00
Michael Eischer
04c23fa95d
rebuild-index: correctly rebuild index for mixed packs
...
For mixed packs, data and tree blobs were stored in separate index
entries. This results in warning from the check command and maybe other
problems.
2022-07-02 19:24:02 +02:00
MichaelEischer
bb5f196b09
Merge pull request #3733 from restic/improve-stats
...
Improve stats
2022-07-02 19:07:31 +02:00
MichaelEischer
c16f989d4a
Merge pull request #3470 from MichaelEischer/sanitize-debug-log
...
Sanitize debug log
2022-07-02 19:00:54 +02:00
Michael Eischer
00d7fcff96
extend compression feature changelog entry
2022-07-02 18:55:59 +02:00
Michael Eischer
a6e9e08034
Account for pack header overhead at each entry
...
This will miss the pack header crypto overhead and the length field,
which only amount to a few bytes per pack file.
2022-07-02 18:55:58 +02:00
Michael Eischer
856d5e4303
stats: return storage size for raw-data mode
...
raw-data summed up the size of the blob plaintexts. However, with
compression this makes little sense as the storage size in the
repository is lower due to compression. Thus sum up the actual size each
blob takes in the repository.
2022-07-02 18:55:12 +02:00
Alexander Neumann
6c4ceaf1e7
Print number of bytes added to the repo
...
This includes optional compression and crypto overhead.
2022-07-02 18:55:12 +02:00
Alexander Neumann
99634c0936
Return real size from SaveBlob
2022-07-02 18:55:12 +02:00
MichaelEischer
fdc53a9d32
Merge pull request #3787 from MichaelEischer/refactor-repository
...
repository: (Mostly) index-related cleanups
2022-07-02 18:54:04 +02:00
Michael Eischer
6923353c43
redact swift auth token in debug output
2022-07-02 18:47:35 +02:00
Michael Eischer
5a11d14082
redacted keys/token in backend config debug log
2022-07-02 18:47:35 +02:00
Michael Eischer
0936d864a4
redact http authorization header in debug log output
2022-07-02 18:47:35 +02:00
Michael Eischer
ec7c9ce88b
drop unused repository.Loader interface
2022-07-02 18:39:59 +02:00
Michael Eischer
2cd7e90ad1
repository: cleanup
2022-07-02 18:39:59 +02:00
Michael Eischer
c1a8fa4290
repository: remove unused packIDToIndex field
2022-07-02 18:39:59 +02:00
Michael Eischer
e68c3a4e62
repository: simplify CreateIndexFromPacks
2022-07-02 18:39:59 +02:00
Michael Eischer
1974ad7ce2
repository: hide MasterIndex.FinalizeFullIndexes / FinalizeNotFinalIndexes
2022-07-02 18:39:59 +02:00
Michael Eischer
ef53ca4a5a
repository: remove MasterIndex.All()
2022-07-02 18:39:59 +02:00
Michael Eischer
bf81bf0795
repository: Properly set id for finalized index
...
As MergeFinalIndex and index uploads can occur concurrently, it is
necessary for MergeFinalIndex to check whether the IDs for an index were
already set before merging it. Otherwise, we'd loose the ID of an index
which is set _after_ uploading it.
2022-07-02 18:39:59 +02:00
Michael Eischer
e0a7852b8b
repository: remove unused (Master)Index.Count
2022-07-02 18:39:58 +02:00
Michael Eischer
8ef2968f28
repository: remove unused index.ListPack
2022-07-02 18:39:12 +02:00
Michael Eischer
e4f20dea61
repository: inline index.encode
2022-07-02 18:39:12 +02:00
Michael Eischer
fe5a8e137a
repository: remove unused index.Store
2022-07-02 18:39:12 +02:00
Michael Eischer
628ae799ca
repository: make flushPacks private
2022-07-02 18:39:12 +02:00
Michael Eischer
ed8aa15376
repository: add Save method to MasterIndex interface
2022-07-02 18:38:56 +02:00
Michael Eischer
a77d5c4d11
repository: index saving belongs into the MasterIndex
2022-07-02 18:38:56 +02:00
MichaelEischer
19641bf828
Merge pull request #3810 from greatroar/revert-3786
...
Revert "restic prune: Merge three loops over the index"
2022-07-01 23:13:39 +02:00
greatroar
a0fa9c6e9f
Revert "restic prune: Merge three loops over the index"
...
This reverts commit 8bdfcf779f
.
Should fix #3809 . Also needed to make #3290 apply cleanly.
2022-06-30 15:27:34 +02:00
greatroar
90d2c0502b
cmd/restic, limiter: Move config knowledge to internal packages
...
The GlobalOptions struct now embeds a backend.TransportOptions, so it
doesn't need to construct one in open and create. The upload and
download limits are similarly now a struct in internal/limiter that is
embedded in GlobalOptions.
2022-06-22 18:29:58 +02:00
MichaelEischer
bc96879d41
Merge pull request #3785 from MichaelEischer/replace-tomb-usage
...
Remove usage of tomb package
2022-06-19 14:42:48 +02:00
MichaelEischer
307f14604f
Merge pull request #3795 from greatroar/sema
...
backend: Move semaphores to a dedicated package
2022-06-18 17:12:01 +02:00
MichaelEischer
19581dbc18
Merge pull request #3786 from greatroar/prune
...
restic prune: Merge three loops over the index
2022-06-18 16:54:50 +02:00
greatroar
8bdfcf779f
restic prune: Merge three loops over the index
...
There were three loops over the index in restic prune, to find
duplicates, to determine sizes (in pack.Size) and to generate packInfos.
These three are now one loop. This way, prune doesn't need to construct
a set of duplicate blobs, pack.Size doesn't need to contain special
logic for prune's use case (the onlyHdr argument) and pack.Size doesn't
need to construct a map only to have it immediately transformed into a
different map.
Some quick testing on a 160GiB local repo doesn't show running time or
memory use of restic prune --dry-run changing significantly.
2022-06-18 10:40:33 +02:00
greatroar
910d917b71
backend: Move semaphores to a dedicated package
...
... called backend/sema. I resisted the temptation to call the main
type sema.Phore. Also, semaphores are now passed by value to skip a
level of indirection when using them.
2022-06-18 10:01:58 +02:00
MichaelEischer
2c893fe43c
Merge pull request #3798 from greatroar/errors
...
all: Move away from pkg/errors, easy cases
2022-06-17 19:01:40 +02:00
greatroar
f92ecf13c9
all: Move away from pkg/errors, easy cases
...
github.com/pkg/errors is no longer getting updates, because Go 1.13
went with the more flexible errors.{As,Is} function. Use those instead:
errors from pkg/errors already support the Unwrap interface used by 1.13
error handling. Also:
* check for io.EOF with a straight ==. That value should not be wrapped,
and the chunker (whose error is checked in the cases changed) does not
wrap it.
* Give custom Error methods pointer receivers, so there's no ambiguity
when type-switching since the value type will no longer implement error.
* Make restic.ErrAlreadyLocked private, and rename it to
alreadyLockedError to match the stdlib convention that error type
names end in Error.
* Same with rest.ErrIsNotExist => rest.notExistError.
* Make s3.Backend.IsAccessDenied a private function.
2022-06-14 08:36:38 +02:00
MichaelEischer
0c0e7b6957
Merge pull request #3776 from wjiec/bugfix/maxkeys-in-search
...
Limit number of key files tested while opening a repository
2022-06-12 15:51:10 +02:00
Michael Eischer
c9ef873192
tweak password test count changelog
2022-06-12 15:39:06 +02:00
Michael Eischer
d08549f0ae
fix flaky key test
2022-06-12 14:19:06 +02:00
Jayson Wang
f144920ed5
fix handling of maxKeys in SearchKey
2022-06-12 14:19:06 +02:00
Alexander Neumann
1dd4b9b60e
Merge pull request #3788 from greatroar/sftp-posix-rename
...
backend/sftp: Support atomic rename
2022-06-06 19:39:48 +02:00
Alexander Neumann
07114ccb21
Merge pull request #3789 from greatroar/fix-loadblob
...
internal/repository: Fix LoadBlob + fuzz test
2022-06-06 19:33:42 +02:00
greatroar
c9557b2822
internal/repository: Fix LoadBlob + fuzz test
...
When given a buf that is big enough for a compressed blob but not its
decompressed contents, the copy at the end of LoadBlob would skip the
last part of the contents.
Fixes #3783 .
2022-06-06 17:02:28 +02:00
greatroar
fa8f02292e
backend/sftp: Support atomic rename
...
... if the server has posix-rename@openssh.com .
OpenSSH introduced this extension in 2008:
7c29661471
2022-06-06 13:40:42 +02:00
Alexander Neumann
7d64aa7f57
Merge pull request #3784 from MichaelEischer/cleanup-migrate-check
...
migrate: Cleanup option to request repository check
2022-06-05 17:37:45 +02:00
Michael Eischer
853ceb3bec
get rid of tomb package
2022-06-05 15:54:57 +02:00
Michael Eischer
e002b09d57
archiver: free workers once finished
2022-06-05 15:48:10 +02:00
Michael Eischer
408ac1a0c2
archiver: remove tomb usage
2022-06-05 15:47:52 +02:00
Michael Eischer
5eba1217e7
migrate: Cleanup option to request repository check
2022-06-04 23:45:00 +02:00
MichaelEischer
0cb6b3d80a
Merge pull request #3778 from greatroar/ellipsis
...
cmd/restic: Remove trailing "..." from progress messages
2022-06-04 19:16:00 +02:00
MichaelEischer
60ca6b1418
Merge pull request #3774 from greatroar/archiver-pool
...
archiver: Remove cleanup goroutine from BufferPool
2022-06-04 18:50:24 +02:00
Jerome Küttner
9adaa6e240
Delete existing path before restoring a symlink
2022-06-01 17:26:25 +02:00
greatroar
b7c990871f
cmd/restic: Remove trailing "..." from progress messages
...
These were added after message since the last refactor of the progress
printing code. Also skips an allocation in the common case.
2022-05-31 19:06:26 +02:00
greatroar
0db1d11b2e
archiver: Remove cleanup goroutine from BufferPool
...
This isn't doing anything. Channels should get cleaned up by the GC when
the last reference to them disappears, just like all other data
structures. Also inlined BufferPool.Put in Buffer.Release, its only
caller.
2022-05-29 17:09:16 +02:00
Alexander Neumann
74f7fe2b98
Merge pull request #3767 from MichaelEischer/fix-prune-empty-snapshot
...
prune: Fix crash on snapshot loading error
2022-05-29 16:52:45 +02:00
Alexander Neumann
d2c5843c68
Merge pull request #3704 from MichaelEischer/compression-migrations
...
Support migration to repository format with compression
2022-05-29 15:52:21 +02:00
Alexander Neumann
78a21bbccf
Merge pull request #3752 from MichaelEischer/fix-dir-sync-errors
...
local: Ignore additional errors for directory syncing
2022-05-29 12:54:51 +02:00
MichaelEischer
2ce8587598
Merge pull request #3771 from greatroar/id-marshaljson
...
internal/restic: Custom ID.MarshalJSON
2022-05-28 16:24:51 +02:00
Michael Eischer
a73fc31b50
Fix linter check
2022-05-28 16:13:46 +02:00
greatroar
dde8e9e296
internal/restic: Custom ID.MarshalJSON
...
This skips an allocation. internal/archiver benchmarks, Linux/amd64:
name old time/op new time/op delta
ArchiverSaveFileSmall-8 3.94ms ± 6% 3.91ms ± 6% ~ (p=0.947 n=20+20)
ArchiverSaveFileLarge-8 304ms ± 3% 301ms ± 4% ~ (p=0.265 n=18+18)
name old speed new speed delta
ArchiverSaveFileSmall-8 1.04MB/s ± 6% 1.05MB/s ± 6% ~ (p=0.803 n=20+20)
ArchiverSaveFileLarge-8 142MB/s ± 3% 143MB/s ± 4% ~ (p=0.421 n=18+19)
name old alloc/op new alloc/op delta
ArchiverSaveFileSmall-8 17.9MB ± 0% 17.9MB ± 0% -0.01% (p=0.000 n=19+19)
ArchiverSaveFileLarge-8 382MB ± 2% 382MB ± 1% ~ (p=0.687 n=20+19)
name old allocs/op new allocs/op delta
ArchiverSaveFileSmall-8 540 ± 1% 528 ± 0% -2.19% (p=0.000 n=19+19)
ArchiverSaveFileLarge-8 1.93k ± 3% 1.79k ± 4% -7.06% (p=0.000 n=20+20)
2022-05-27 12:26:37 +02:00
Alexander Neumann
bc27c370e7
Update gopkg.in/yaml
...
This fixes a panic in invalid input, but I think we aren't affected.
2022-05-26 14:23:49 +02:00
Alexander Neumann
9e30152f3c
Merge pull request #3770 from lbausch/update-minio-go
...
Update github.com/minio/minio-go/v7 to v7.0.27
2022-05-26 14:21:19 +02:00
Lorenz Bausch
cc3f8d3732
Update github.com/minio/minio-go/v7 to v7.0.27
...
This version adds support for Cloudflare R2, as discussed in #3757
2022-05-26 13:05:13 +02:00
Michael Eischer
c8e1ac4049
prune: Don't print stack trace if snapshot can't be loaded
2022-05-23 22:38:45 +02:00
Michael Eischer
173695104c
prune: Fix crash on empty snapshot
2022-05-23 22:32:59 +02:00
MichaelEischer
ded783dd61
Merge pull request #3762 from marigbede/marigbede-gs-documentation-fix
...
Update 030_preparing_a_new_repo.rst
2022-05-23 22:10:38 +02:00
Arigbede Moses
2aa75d6272
Update 030_preparing_a_new_repo.rst
2022-05-18 22:03:59 +01:00
MichaelEischer
88a8701fb5
Merge pull request #3734 from lbausch/validate-patterns
...
Validate exclude patterns
2022-05-14 16:20:15 +02:00
MichaelEischer
b2a2e5f727
Merge pull request #3753 from greatroar/indexmap-alloc
...
repository: Re-tune indexmap allocation strategy
2022-05-14 15:44:08 +02:00
MichaelEischer
b52c631bd3
Merge pull request #3754 from greatroar/simplify-hashing
...
hashing: Fix up comments
2022-05-14 15:33:51 +02:00
Lorenz Bausch
e7df66cc91
Add changelog entry for validating exclude patterns
2022-05-11 22:41:05 +02:00
Lorenz Bausch
36bd464e8c
Add tests for validating exclude patterns
2022-05-11 22:41:00 +02:00
greatroar
39a335e690
hashing: Fix up comments
2022-05-11 21:36:10 +02:00
greatroar
5141228e0c
repository: Re-tune indexmap allocation strategy
...
fd05037e1a
changed the allocation batch
size from 256 to 128 under the assumption that an indexEntry is 60 bytes
on amd64, but it's 64: structs are padded out to a multiple of 8 for
alignment reasons. That means we'd waste no space in malloc even without
the batch allocation, at least on 64-bit machines. While that strategy
cuts the overallocation down dramatically for many small indexes, it also
seems to slow allocation down (Go 1.18, Linux, amd64, -benchtime=2s):
name old time/op new time/op delta
DecodeIndex-8 4.67s ± 5% 4.60s ± 1% ~ (p=0.953 n=10+5)
DecodeIndexParallel-8 4.67s ± 3% 4.60s ± 1% ~ (p=0.953 n=10+5)
IndexHasUnknown-8 37.8ns ± 8% 36.5ns ±14% ~ (p=0.841 n=5+5)
IndexHasKnown-8 38.5ns ±12% 37.7ns ±10% ~ (p=0.968 n=5+5)
IndexAlloc-8 615ms ±18% 607ms ± 1% ~ (p=1.000 n=10+5)
IndexAllocParallel-8 245ms ±11% 285ms ± 6% +16.40% (p=0.001 n=10+5)
MasterIndexAlloc-8 286ms ± 9% 275ms ± 2% ~ (p=1.000 n=10+5)
LoadIndex/v1-8 27.0ms ± 4% 26.8ms ± 1% ~ (p=0.690 n=5+5)
LoadIndex/v2-8 22.4ms ± 1% 22.8ms ± 2% +1.48% (p=0.016 n=5+5)
name old alloc/op new alloc/op delta
IndexAlloc-8 446MB ± 0% 446MB ± 0% -0.00% (p=0.000 n=8+4)
IndexAllocParallel-8 446MB ± 0% 446MB ± 0% -0.00% (p=0.008 n=8+5)
MasterIndexAlloc-8 213MB ± 0% 159MB ± 0% -25.47% (p=0.000 n=10+5)
name old allocs/op new allocs/op delta
IndexAlloc-8 913k ± 0% 2632k ± 0% +188.19% (p=0.008 n=5+5)
IndexAllocParallel-8 913k ± 0% 2632k ± 0% +188.21% (p=0.008 n=5+5)
MasterIndexAlloc-8 318k ± 0% 1172k ± 0% +267.86% (p=0.008 n=5+5)
Instead, this patch sets a batch size of 4, which means no space is
wasted by malloc on 64-bit and very little on 32-bit. It still gets very
close to the savings from not allocating in batches, without requiring
special code for bits.UintSize==64. Benchmark results, again for
Linux/amd64:
name old time/op new time/op delta
DecodeIndex-8 4.67s ± 5% 4.83s ± 9% ~ (p=0.315 n=10+10)
DecodeIndexParallel-8 4.67s ± 3% 4.68s ± 4% ~ (p=0.315 n=10+10)
IndexHasUnknown-8 37.8ns ± 8% 44.5ns ±19% ~ (p=0.095 n=5+5)
IndexHasKnown-8 38.5ns ±12% 36.9ns ± 8% ~ (p=0.690 n=5+5)
IndexAlloc-8 615ms ±18% 628ms ±18% ~ (p=0.218 n=10+10)
IndexAllocParallel-8 245ms ±11% 262ms ± 9% +7.02% (p=0.043 n=10+10)
MasterIndexAlloc-8 286ms ± 9% 287ms ±13% ~ (p=1.000 n=10+10)
LoadIndex/v1-8 27.0ms ± 4% 26.8ms ± 0% ~ (p=1.000 n=5+5)
LoadIndex/v2-8 22.4ms ± 1% 22.5ms ± 0% ~ (p=0.056 n=5+5)
name old alloc/op new alloc/op delta
IndexAlloc-8 446MB ± 0% 446MB ± 0% ~ (p=1.000 n=8+10)
IndexAllocParallel-8 446MB ± 0% 446MB ± 0% -0.00% (p=0.000 n=8+8)
MasterIndexAlloc-8 213MB ± 0% 160MB ± 0% -25.02% (p=0.000 n=10+9)
name old allocs/op new allocs/op delta
IndexAlloc-8 913k ± 0% 1333k ± 0% +45.94% (p=0.000 n=8+10)
IndexAllocParallel-8 913k ± 0% 1333k ± 0% +45.94% (p=0.000 n=8+8)
MasterIndexAlloc-8 318k ± 0% 525k ± 0% +64.99% (p=0.000 n=10+10)
The allocation method indexmap.newEntry has also been rewritten in a
form that is a few instructions shorter.
2022-05-11 21:22:14 +02:00
Michael Eischer
48a0d83143
local: Ignore additional errors for directory syncing
...
Apparently SMB/CIFS on Linux/macOS returns somewhat random errnos when
trying to sync a windows share which does not support calling fsync for
a directory.
2022-05-11 20:37:59 +02:00
MichaelEischer
ac36fda155
Merge pull request #3749 from greatroar/simplify-hashing
...
hashing: Remove io.WriterTo implementation
2022-05-11 20:03:43 +02:00
MichaelEischer
df554e5f69
Merge pull request #3748 from greatroar/runworkers
...
repository: Remove RunWorkers, report ctx.Err()
2022-05-11 19:38:46 +02:00
greatroar
54b8337813
hashing: Remove io.WriterTo implementation
...
This functionality has gone unused since
4b3dc415ef
changed hashing.Reader's only
client to use ioutil.ReadAll on a bufio.Reader wrapping the hashing
Reader.
Reverts bcb852a8d0
.
2022-05-10 23:41:18 +02:00
greatroar
2e0f1f5113
repository: Remove RunWorkers, report ctx.Err()
...
This removes RunWorkers, which had become mere overhead by successive
refactors. It also ensures that each former user of that function
returns any context error that occurs, so failure to complete an
operation is always reported as an error.
2022-05-10 22:26:00 +02:00
MichaelEischer
47c56dea5c
Merge pull request #3746 from greatroar/cache-lstat
...
cache: Don't Lstat before creating CACHEDIR.TAG
2022-05-10 20:31:15 +02:00
Alexander Neumann
c270ab1e08
Merge pull request #3744 from MichaelEischer/fix-windows-temp-file
...
Fix error on temp file deletion on windows
2022-05-10 19:53:25 +02:00
greatroar
2da377c582
cache: Don't Lstat before creating the tag file
...
The tag file is opened with O_CREATE|O_EXCL and ErrExist is handled, so
we don't need to check for existence first.
2022-05-10 18:52:39 +02:00
Michael Eischer
ae7e51382a
Fix error on temp file deletion on windows
...
Apparently it can take a moment between closing a tempfile marked as
DELETE_ON_CLOSE and it actually being deleted. During that time the file
is inaccessible. Thus just skip deleting the temp file on windows.
2022-05-09 22:43:26 +02:00
Michael Eischer
5c6db534d4
Add compression migration support to changelog
2022-05-09 22:39:02 +02:00
Michael Eischer
c1bbbcd0dc
migrate: Allow migrations to request a check run
...
This is currently only used by upgrade_repo_v2.
2022-05-09 22:31:30 +02:00
Michael Eischer
59eb132dcd
check: Better differentiate between warnings and errors
2022-05-09 22:31:30 +02:00
Michael Eischer
5815f727ee
checker: convert error type to use pointer-receivers
2022-05-09 22:31:30 +02:00
Michael Eischer
4faff0debe
doc: Describe repository upgrade process
2022-05-09 22:31:30 +02:00
Michael Eischer
e36a40db10
upgrade_repo_v2: Use atomic replace for supported backends
2022-05-09 22:31:30 +02:00
Michael Eischer
7559d2f105
Document repository version and minimum restic version
2022-05-09 22:31:30 +02:00
Michael Eischer
381bd94c6c
prune: Add option to repack uncompressed data
2022-05-09 22:31:30 +02:00
Michael Eischer
5406743102
prune: Automatically repack uncompressed trees for repo v2
...
Tree packs are cached locally at clients and thus benefit a lot from
being compressed. Ensure this be having prune always repack pack files
containing uncompressed trees.
2022-05-09 22:31:30 +02:00
Alexander Neumann
c8c0d659ec
Add migration to compress all data
2022-05-09 22:31:30 +02:00
Alexander Neumann
8c244214bf
Add tests for upgrade migration
2022-05-09 22:31:30 +02:00
Alexander Neumann
a5f1d318ac
Try to make repo upgrade migration more failsafe
2022-05-09 22:31:30 +02:00
Alexander Neumann
82ed5a3a15
Add repo upgrade migration
2022-05-09 22:31:30 +02:00
Alexander Neumann
3af6c180e4
Improve migrate command
2022-05-09 22:31:30 +02:00
Michael Eischer
92816fa966
init: Enable compression support by default
2022-05-09 22:31:30 +02:00
MichaelEischer
ab49c14621
Merge pull request #3740 from MichaelEischer/fix-restore-size-stats
...
Fix restore size calculation for multiple snapshots
2022-05-09 21:39:45 +02:00
Michael Eischer
2c07f7fff3
stats: hardlinks only reduce restore within a snapshot
...
The `stats` command checks inodes to not count hardlinked files multiple
times into the restore size. This check applies across all snapshots and
not only within snapshots. As a result the result size was far too low
when calculating it for multiple snapshots and it would vary depending
on the order in which snapshots were listed.
2022-05-09 21:26:24 +02:00
Lorenz Bausch
9fb81c4246
Validate exclude patterns
2022-05-07 21:12:47 +02:00
Lorenz Bausch
e7fd200237
Keep original pattern for later use
2022-05-07 21:08:09 +02:00
MichaelEischer
cc8a03b1d0
Merge pull request #3735 from HenrikBengtsson/master
...
DOCS: Incorrectly used SI units when IEC units were meant (fix #3669 )
2022-05-07 11:15:27 +02:00
Henrik Bengtsson
9bb532672a
DOCS: Incorrectly used SI units when IEC units were meant ( fix #3669 )
2022-05-02 21:21:39 -07:00
MichaelEischer
26c333325c
Merge pull request #3715 from ema/master
...
doc: specify AWS Region via AWS_DEFAULT_REGION
2022-05-02 21:46:56 +02:00
Emanuele Rocca
38c0531b52
doc: specify AWS Region via AWS_DEFAULT_REGION
...
If no specific Region is mentioned in RESTIC_REPOSITORY, AWS defaults to
us-east-1. For this reason, users that follow the tutorial and create
their S3 bucket in any other region get the following error:
"Fatal: create repository at [...] client.BucketExists"
Explicitly specifying the AWS region name fixes the issue.
2022-05-02 21:26:58 +02:00
Alexander Neumann
fb5b9370f3
Merge pull request #3728 from MichaelEischer/debug-examine-upload
...
Add support to reupload blobs in `debug examine`
2022-04-30 20:39:19 +02:00
Alexander Neumann
ffbd48c0c6
Merge pull request #3481 from MichaelEischer/recover-enospace
...
Recover from no free space errors
2022-04-30 20:22:26 +02:00
Michael Eischer
95bcc9ea31
debug: Support pack ID prefixes in `debug examine`
2022-04-30 20:20:31 +02:00
Michael Eischer
2d6a943911
debug: Add switch to upload blobs extracted by debug examine
...
This simplifies salvaging a damaged pack file. Reuploading tree blobs
was previously not possible.
2022-04-30 20:20:31 +02:00
Alexander Neumann
9af499d8a4
Tidy go.sum
2022-04-30 20:16:09 +02:00
Alexander Neumann
2e3d23c1d7
Update github.com/klauspost/compress
2022-04-30 20:03:21 +02:00
Michael Eischer
dbbeac7174
prune: Add unsafe option to recover from no free space
...
The new option allows prune to operate with nearly no scratch space by only removing
no longer necessary pack files and first deleting the index before
rebuilding it. By first deleting the index it becomes safe to just
delete no longer necessary pack files. However, as a downside there's
now the risk that the repository becomes inaccessible if prune fails.
To recover from that problem a user might have to manually delete the
repository index and then run (a full) `rebuild-index` again.
2022-04-30 19:21:07 +02:00
Michael Eischer
cf5cb673fb
repository: Use existing method to collect pack ids
2022-04-30 19:14:21 +02:00
Michael Eischer
b335cb6285
repository: Refactor index IDs collection
2022-04-30 19:14:21 +02:00
MichaelEischer
9c047f170a
Merge pull request #3419 from DanielG/relax-file-modes
...
Enable admin to decide group access to repository files
2022-04-30 16:39:37 +02:00
Daniel Gröber
f31b4f29c1
Use config file modes to derive new dir/file modes
...
Fixes #2351
2022-04-30 15:59:51 +02:00
Alexander Neumann
71c653f9e0
Merge pull request #3727 from MichaelEischer/changelog-3475
...
Add changelog for local/sftp connection limit
2022-04-30 14:05:21 +02:00
Michael Eischer
29a8f92967
Add changelog for local/sftp connection limit
2022-04-30 13:36:47 +02:00
MichaelEischer
ac9324aeaf
Merge pull request #3666 from MichaelEischer/compression
...
Implement compression support
2022-04-30 11:49:05 +02:00
Alexander Neumann
dc5adef255
Add documentation for --repository-version
2022-04-30 11:34:10 +02:00
Michael Eischer
4b01b06f2f
repository: Test compressed blobs in StreamPack
2022-04-30 11:34:10 +02:00
Michael Eischer
bcab548617
pack: slightly expand testing of compressed blobs
2022-04-30 11:34:10 +02:00
Michael Eischer
ec2b25565a
repository: test uncompressedLength field and index example
2022-04-30 11:34:10 +02:00
Michael Eischer
9ffb8920f1
repository: run blackbox tests using old and new repo version
2022-04-30 11:34:10 +02:00
Michael Eischer
abe5935693
repository: unify repository version-specific initialization
...
Mark the master index as compressed also when initializing a new
repository. This is only relevant for testing.
2022-04-30 11:34:10 +02:00
Alexander Neumann
8776031f96
Leave allocating slices to the decompress code
2022-04-30 11:34:10 +02:00
Alexander Neumann
5eb05a0afe
Configure zstd encoder/decoder
2022-04-30 11:34:10 +02:00
Michael Eischer
2f36e044db
Cleanup pack header check
2022-04-30 11:34:10 +02:00
Alexander Neumann
94dc9a0fa7
Amend changelog
2022-04-30 11:34:10 +02:00
Alexander Neumann
8b11b86383
Add option global --compression
2022-04-30 11:34:10 +02:00
Michael Eischer
f38f457a64
Add basic changelog for compression support
2022-04-30 11:34:10 +02:00
Michael Eischer
ba27d29d58
Print repository version when opening a repo
2022-04-30 11:34:10 +02:00
Michael Eischer
7132df529e
repository: Increase index size for repo version 2
...
A compressed index is only about one third the size of an uncompressed
one. Thus increase the number of entries in an index to avoid cluttering
the repository with small indexes.
2022-04-30 11:34:10 +02:00
Michael Eischer
2535524132
debug: Add support for compressed blobs
2022-04-30 11:34:10 +02:00
Michael Eischer
fda7bb0f09
debug: Reduce code duplication
2022-04-30 11:34:10 +02:00
Michael Eischer
66f9048bce
repository: Alloc zstd encoder/decoder on demand
2022-04-30 11:34:10 +02:00
Michael Eischer
fd05037e1a
repository: recalibrate index batch allocation size
2022-04-30 11:34:10 +02:00
Michael Eischer
6fb408d90e
repository: implement pack compression
2022-04-30 11:34:10 +02:00
Michael Eischer
362ab06023
init: Add flag to specify created repository version
2022-04-30 10:07:42 +02:00
Michael Eischer
4b957e7373
repository: Implement index/snapshot/lock compression
...
The config file is not compressed as it should remain readable by older
restic versions such that these can return a proper error.
As the old format for unpacked data does not include a version header,
make use of a trick: The old data is always encoded as JSON. Thus it can
only start with '{' or '['. For any other value the first byte indicates
a versioned format. The version is set to 2 for now. Then the zstd
compressed data follows.
2022-04-30 10:07:42 +02:00
Michael Eischer
0957b74887
Misc design.rst cleanups
2022-04-30 10:07:42 +02:00
Alexander Neumann
270ed00d1f
doc: Add repository compression support documentation
...
Co-authored-by: Michael Eischer <michael.eischer@fau.de>
2022-04-30 10:07:42 +02:00
Alexander Neumann
4e1ef7804a
Merge pull request #3717 from MichaelEischer/fix-stuck-repack
...
Fix stuck repack step
2022-04-30 09:50:43 +02:00
Alexander Neumann
e4780d3956
Merge pull request #3718 from MichaelEischer/sftp-docs-fix
...
doc: sftp with password actually works
2022-04-23 20:27:27 +02:00
Alexander Neumann
c183e35b5a
Merge pull request #3719 from MichaelEischer/read-write-order
...
Describe repository read/write order required by repository format
2022-04-23 15:23:00 +02:00
Michael Eischer
6f9e20a1bb
doc: Describe repository read/write order
2022-04-23 15:21:02 +02:00
Michael Eischer
f9219e8608
doc: sftp with password actually works
2022-04-23 11:57:36 +02:00
Michael Eischer
3b630d9998
add missing streamPacks changelog
2022-04-23 11:50:19 +02:00
Michael Eischer
566ac11c65
fix changelog name
2022-04-23 11:37:00 +02:00
Michael Eischer
f5609d1d3c
prune: Fail early if too few backend connections
2022-04-23 11:32:52 +02:00
Michael Eischer
e597b99b55
repository: Reduce repack workers to prevent deadlock
...
As repack streams packs these occupy one backend connection. Uploading a
new pack also requires a backend connection. To prevent a deadlock
during repack when reaching the backend connections limit, simply limit
the repackWorker count to always leave one connection for uploading.
2022-04-23 11:28:18 +02:00
Michael Eischer
ee627cd832
backend/mem: Actually enforce connection limit
...
This will allow tests to detect deadlocks related to the connections
limit.
2022-04-23 11:22:00 +02:00
Michael Eischer
4f97492d28
Backend: Expose connections parameter
2022-04-23 11:13:08 +02:00
rawtaz
07a565e6f7
Merge pull request #3716 from MichaelEischer/password-error-on-stderr
...
Print password error message on stderr
2022-04-21 01:40:54 +02:00
Michael Eischer
bf7da7ff10
Print password error message on stderr
...
The password prompt itself is already printed on stderr.
2022-04-20 22:22:09 +02:00
Alexander Neumann
dba47d29d5
Merge pull request #3711 from restic/doc-grouping
...
doc: Clarify and make grouping in forget more noticeable
2022-04-16 10:08:11 +02:00
Leo R. Lundgren
8ac7519fd5
doc: Clarify and make grouping in forget more noticeable
2022-04-16 01:13:13 +02:00
Alexander Neumann
edc1a24a90
Merge pull request #3707 from duracell/patch-1
...
doc: fix missing "init" in rest-server example
2022-04-12 20:39:25 +02:00
Michael
9563e2f75c
doc: fix missing "init" in rest-server example
2022-04-12 18:21:53 +02:00
Alexander Neumann
7f133a28b2
Update VERSION file
2022-04-11 20:34:14 +02:00
Alexander Neumann
4f3b1f19cb
Set development version for 0.13.1
2022-04-10 20:41:00 +02:00
Alexander Neumann
89ee1cf9ee
Merge pull request #3610 from MichaelEischer/windows-temp-files
...
Improve handling of temporary files on windows
2022-04-10 20:29:04 +02:00
Alexander Neumann
a059ef90f8
Merge pull request #3702 from MichaelEischer/extend-config-error
...
Print used key name if config fails to load
2022-04-10 20:25:24 +02:00
Michael Eischer
4077a81b34
Add simple test for fs.TempFile on windows
2022-04-09 23:37:58 +02:00
Michael Eischer
9a3f1a9703
Simplify and comment TempFile implementation for windows
2022-04-09 23:37:58 +02:00
Michael Eischer
c2aabb2686
Print used key name if config fails to load
2022-04-09 22:38:18 +02:00
MichaelEischer
c60a5f00c9
Merge pull request #3675 from ItsMattL/update
...
Refactor file handing for self-update.
2022-04-09 21:55:56 +02:00
Matt LaPlante
0ba9d4ced7
Refactor file handing for self-update.
...
* Write new file payload to a temp file before touching the original
binary. Minimizes the possibility of failing mid-write and corrupting
the binary.
* On Windows, move the original binary out to a temp file rather than
removing it as the running binary is locked. Fixes issue #2248 .
2022-04-09 21:40:33 +02:00
Alexander Neumann
04e054465a
Merge pull request #3475 from MichaelEischer/local-sftp-conn-limit
...
Limit concurrent operations for local / sftp backend
2022-04-09 21:33:00 +02:00
Alexander Neumann
1519e9f911
Merge pull request #3570 from MichaelEischer/list-snapshots-before-index
...
List snapshots before index
2022-04-09 21:15:49 +02:00
Michael Eischer
ebab35581c
Check in integration test that snapshots are listed before the index
...
As an exception prune is still allowed to load the index before
snapshots, as it uses exclusive locks. In case of problems with locking
it is also better to load snapshots created after loading the index, as
this will lead to a prune sanity check failure instead of a broken snapshot.
2022-04-09 12:27:27 +02:00
Michael Eischer
7b9ae91e04
copy: Load snapshots before indexes
2022-04-09 12:27:25 +02:00
Michael Eischer
47243176fa
diff: list snapshots only once
2022-04-09 12:26:31 +02:00
Michael Eischer
5af828e3e6
add changelogs
2022-04-09 12:26:31 +02:00
Michael Eischer
4636c20397
test that TestFindListOnce calls List only once
2022-04-09 12:26:31 +02:00
Michael Eischer
9e12159230
Fix O(n) backend list calls in FindFilteredSnapshots
...
When resolving snapshotIDs in FindFilteredSnapshots either
FindLatestSnapshot or FindSnapshot is called. Both operations issue a
list operation to the backend. When for example passing a long list of
snapshot ids to `forget` this could lead to a large number of list
operations.
2022-04-09 12:26:31 +02:00
Michael Eischer
3d29083e60
copy/find/ls/recover/stats: Memorize snapshot listing before index
...
These commands filter the snapshots according to some criteria which
essentially requires loading the index before filtering the snapshots.
Thus create a copy of the snapshots list beforehand and use it later on.
2022-04-09 12:26:30 +02:00
Michael Eischer
2ec0f3303a
backup/diff/dump/restore/stats: List snapshots before index
...
During a backup the index is written before the corresponding snapshots.
To ensure that a concurrent/later restic run can read a snapshot's data,
restic thus must first load the snapshots and only afterwards the index.
Otherwise it is not possible to ensure that the loaded index is recent
enough to cover all of the snapshot's data.
2022-04-09 12:24:09 +02:00
Michael Eischer
ece06f125e
sftp: Limit concurrent backend operations
2022-04-09 12:21:38 +02:00
Michael Eischer
cd783358d3
local: Limit concurrent backend operations
...
Use a limit of 2 similar to the filereader concurrency in the archiver.
2022-04-09 12:21:38 +02:00
Michael Eischer
0b258cc054
backends: clean reader closing
2022-04-09 12:21:38 +02:00
Alex Duchesne
9e34c791c9
Better temp file cleanup on Windows.
2022-04-09 12:00:22 +02:00
Alexander Neumann
7d55b4f95e
Merge pull request #3701 from restic/rawtaz-doc-exclude-cachedir
...
doc: Link to CACHEDIR.TAG specification
2022-04-08 09:35:08 +02:00
rawtaz
de4e3117eb
doc: Link to CACHEDIR.TAG specification
2022-04-08 03:07:27 +02:00
MichaelEischer
500079d265
Merge pull request #3689 from brightdroid/patch-1
...
added documentation for zsh autocompletion
2022-04-03 21:49:47 +02:00
Alexander Neumann
192288bc9c
Merge pull request #3696 from cqjjjzr/fix-win-procgrp
...
Fix rclone (scoop shim) and sftp issue due to detached console on Windows
2022-04-03 13:46:28 +02:00
Charlie Jiang
d9c9415cfd
Fix rclone (scoop shim) and sftp issue due to detached console on Windows
2022-04-03 17:53:17 +08:00
Alexander Neumann
59370b6062
Merge pull request #3695 from jernej-9/typo-fix
...
Fix a typo in the docs
2022-04-01 20:10:40 +02:00
Jernej Debevc
6e5731bf2f
Fix a typo in the docs
2022-04-01 17:16:55 +02:00
Alexander Neumann
305cd1e730
Merge pull request #3693 from greatroar/cast-btrfs-super-magic
...
Cast unix.Statfs_t.Type to int64 when checking for btrfs
2022-04-01 08:09:46 +02:00
greatroar
c23c0f7c14
Cast unix.Statfs_t.Type to int64 when checking for btrfs
...
Fixes #3687 . Uses the cast suggested by @MichaelEischer, except that the
contant isn't cast along, because it's untyped and will be converted by
the compiler as necessary.
2022-03-31 22:30:45 +02:00
Alexander Neumann
774c2e75ca
Merge pull request #3680 from restic/update-deps
...
Update all dependencies (except fuse), require Go 1.15
2022-03-30 21:36:08 +02:00
Alexander Neumann
66d50b72e3
Require Go 1.15 or later
...
The library github.com/golang-jwt/jwt/v4 requires the FillByte() method
of *big.Int, so we're raising the minimum Go version to 1.15.
2022-03-30 21:11:17 +02:00
Alexander Neumann
89d86a7933
Update all dependencies (except fuse)
2022-03-30 21:11:13 +02:00
Christoph Roeder
7510bdc247
added documentation for zsh autocompletion
2022-03-30 10:45:52 +02:00
Alexander Neumann
f190d2e60e
Merge pull request #3686 from restic/fix-diff
...
Fix diff
2022-03-30 07:43:07 +02:00
Alexander Neumann
206550a246
Fix diff
...
Nodes in trees were always printed with a `+` in diff, regardless of
whether or not a dir was added or removed. Let's use the mode we were
passed in printDir().
Closes #3685
2022-03-29 21:05:11 +02:00
Alexander Neumann
db8a958991
Merge pull request #3683 from MichaelEischer/fix-golangci-lint-warnings
...
Fix golangci lint warnings
2022-03-29 11:45:10 +02:00
Alexander Neumann
724ace0e99
Merge pull request #3682 from MichaelEischer/refactor-code
...
Pack size calculation cleanup and misc other changes
2022-03-29 11:06:04 +02:00
Michael Eischer
af31266b7d
golangci-lint: replace deprecated golint with revive
2022-03-28 22:33:17 +02:00
Michael Eischer
2f81af6afa
bloblru: Fix comment for New function
2022-03-28 22:25:25 +02:00
Michael Eischer
61e179ee78
switch to golang.org/x/term
2022-03-28 22:24:15 +02:00
Michael Eischer
c60540b196
add go:build headers everywhere
2022-03-28 22:23:47 +02:00
Michael Eischer
fefe9f5c0e
pack: Hide more implementation details
2022-03-28 22:12:16 +02:00
Michael Eischer
a773cb6527
pack: cleanup header size calculation
2022-03-28 22:09:49 +02:00
Michael Eischer
6408686973
repository: Simplify Blob equality check
2022-03-28 22:09:49 +02:00
Michael Eischer
243698680a
crypto: Use helpers for size calculations
2022-03-28 22:09:49 +02:00
Michael Eischer
d6db5a1fc2
archiver: Fix test
...
The test relied on an undeocumented sideeffect of the LoadBlob implementation
2022-03-28 22:09:49 +02:00
Michael Eischer
f78bd14e28
repository: Remove pack implementation details from MasterIndex
2022-03-28 22:09:49 +02:00
Michael Eischer
dc3d77dacc
repository: make saveAndEncrypt private
2022-03-28 22:09:49 +02:00
Michael Eischer
6877e7edbb
repository: Rename LoadAndDecrypt to LoadUnpacked
...
The method is the complement for SaveUnpacked and not for
SaveAndEncrypt. The latter assembles blobs into pack files.
2022-03-28 22:09:49 +02:00
Michael Eischer
2e1613d4c6
errors: Ensure that errors.IsFatal(errors.Fatal("err")) == true
...
This fixes a few cases where restic output "Fatal: Fatal: [...]"
2022-03-28 22:09:49 +02:00
Alexander Neumann
a08b95c497
Merge pull request #3513 from MichaelEischer/fast-copy
...
Speed-up copy command
2022-03-28 20:18:43 +02:00
Michael Eischer
537b4c310a
copy: Implement by reusing repack
...
The repack operation copies all selected blobs from a set of pack files
into new pack files. For prune the source and destination repositories
are identical. To implement copy, just use a different source and
destination repository.
2022-03-26 20:47:15 +01:00
Alexander Neumann
4d5db61bd0
Merge pull request #3484 from MichaelEischer/stream-check-repack
...
Stream packs in `check --read-data` and during repacking
2022-03-26 20:46:17 +01:00
Alexander Neumann
4ab12f59a5
Set development version for 0.13.0
2022-03-26 20:10:07 +01:00
Alexander Neumann
40791fff64
Add version for 0.13.0
2022-03-26 20:09:59 +01:00
Alexander Neumann
a53a4a23fd
Update manpages and auto-completion
2022-03-26 20:09:59 +01:00
Alexander Neumann
b567c08e80
Generate CHANGELOG.md for 0.13.0
2022-03-26 20:09:40 +01:00
Alexander Neumann
0ca89b6fec
Prepare changelog for 0.13.0
2022-03-26 20:09:39 +01:00
Alexander Neumann
d7e46c187a
Merge pull request #3678 from restic/changelogs
...
Review, reword and polish unreleased changelog entries
2022-03-26 19:44:30 +01:00
Alexander Neumann
6aefe3e111
Merge pull request #3677 from restic/3490-polish
...
check: Adjust help and documentation for check --read-data-subset
2022-03-26 19:34:51 +01:00
Leo R. Lundgren
03137a34db
Review, reword and polish unreleased changelog entries
2022-03-26 13:01:53 +01:00
Leo R. Lundgren
c7d637ec39
check: Adjust help and documentation for check --read-data-subset
2022-03-26 00:11:04 +01:00
Alexander Neumann
6087c4ad75
Merge pull request #3656 from lgommans/forget-security
...
forget: Update docs for readability and append-only considerations
2022-03-24 21:36:19 +01:00
Leo R. Lundgren
cdf478c8f4
doc: More updates to forget documentation and security considerations
2022-03-23 23:12:19 +01:00
Luc Gommans
80969a6347
Update docs according to comments from MichaelEischer in PR #3656
2022-03-23 23:12:19 +01:00
Leo R. Lundgren
676d5d498c
doc: Update forget security considerations and thread model
2022-03-23 23:12:19 +01:00
Luc Gommans
9c1d49e312
Document "forget" security considerations and add references
...
Removing data based on a policy when the attacker had the opportunity to
add data to your repository comes with some considerations. This is
added to the 060_forget.rst documentation.
That document is also updated to reflect that restic now considers
the current system time while running "forget".
References to the security considerations section are added:
- In `restic forget --help`
- In the threat model (design.rst)
- In the (030) setup section where an append-only setup is referenced
A reference is also to be added to the `rest-server` readme's
append-only paragraph (see my fork).
This commit also resolves a typo (amount->number for countable noun),
changes a password length recommendation into the metric that
actually matters when creating passwords (entropy) since I was editing
these doc files anyway, and updates the outdated copyright year in
`conf.py`.
Some wording in 060_forget (line 21..22) was changed to clarify what
"forget" and "prune" do, to try and avoid the apparent misconception
that "forget" does not remove any data.
2022-03-23 23:12:19 +01:00
Alexander Neumann
e682f7c0d6
Add tests for StreamPack
2022-03-21 21:15:03 +01:00
Alexander Neumann
ca1e2316cf
Merge pull request #3665 from MichaelEischer/sane-list-locks
...
list: Never lock the repository when listing lock files
2022-03-21 11:14:44 +01:00
Alexander Neumann
0b8b524f12
Merge pull request #3512 from MichaelEischer/cleaner-lock-refresh
...
Prevent lock refresh from leaving behind lots of stale locks
2022-03-21 11:10:37 +01:00
Alexander Neumann
a350625554
Merge pull request #3524 from MichaelEischer/atomic-sftp
...
sftp: Implement atomic uploads
2022-03-21 11:08:22 +01:00
Alexander Neumann
32e61f2620
Update changelog/unreleased/issue-1106
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2022-03-21 11:04:04 +01:00
Alexander Neumann
8388f66c4c
Merge pull request #3668 from greatroar/symlink-size
...
Report symlink sizes from FUSE mount
2022-03-21 11:02:32 +01:00
Alexander Neumann
0937008648
Merge pull request #3654 from MichaelEischer/limit-huge-tree-streams
...
Limit number of large tree blobs loaded in parallel by StreamTrees
2022-03-21 11:01:04 +01:00
Alexander Neumann
3a285f91bc
Merge pull request #2311 from vincentbernat/feature/negative-pattern
...
filter: ability to use negative patterns
2022-03-20 14:02:30 +01:00
Alexander Neumann
29a5778626
Improve wording
2022-03-20 13:46:16 +01:00
Michael Eischer
53656f019a
filter: address review comments
2022-03-20 13:33:08 +01:00
Michael Eischer
cd190bee14
filter: short circuit if no negative patterns
2022-03-20 13:33:08 +01:00
Vincent Bernat
2ee07ded2b
filter: ability to use negative patterns
...
This is quite similar to gitignore. If a pattern is suffixed by an
exclamation mark and match a file that was previously matched by a
regular pattern, the match is cancelled. Notably, this can be used
with `--exclude-file` to cancel the exclusion of some files.
Like for gitignore, once a directory is excluded, it is not possible
to include files inside the directory. For example, a user wanting to
only keep `*.c` in some directory should not use:
~/work
!~/work/*.c
But:
~/work/*
!~/work/*.c
I didn't write documentation or changelog entry. I would like to get
feedback if this is the right approach for excluding/including files
at will for backups. I use something like this as an exclude file to
backup my home:
$HOME/**/*
!$HOME/Documents
!$HOME/code
!$HOME/.emacs.d
!$HOME/games
# [...]
node_modules
*~
*.o
*.lo
*.pyc
# [...]
$HOME/code/linux/*
!$HOME/code/linux/.git
# [...]
There are some limitations for this change:
- Patterns are not mixed accross methods: patterns from file are
handled first and if a file is excluded with this method, it's not
possible to reinclude it with `--exclude !something`.
- Patterns starting with `!` are now interpreted as a negative
pattern. I don't think anyone was relying on that.
- The whole list of patterns is walked for each match. We may
optimize later by exiting early if we know no pattern is starting
with `!`.
Fix #233
2022-03-20 13:33:08 +01:00
Michael Eischer
12606b575f
filter: Cleanup variable naming
2022-03-20 13:33:08 +01:00
Michael Eischer
5f145f0c7e
filter: introduce pattern struct
2022-03-20 13:33:08 +01:00
Vincent Bernat
13c40d4199
filter: additional tests for filter.List()
2022-03-20 13:33:08 +01:00
Alexander Neumann
13aae82635
Merge pull request #3673 from restic/update-go
...
Update go
2022-03-20 12:02:32 +01:00
Alexander Neumann
b85d035956
Fix calens install step
2022-03-20 11:36:45 +01:00
Alexander Neumann
47aa4613f7
Force Go to use Module Mode
2022-03-20 11:30:01 +01:00
Alexander Neumann
a9a5acb8ce
Update golangci-lint
2022-03-20 11:26:56 +01:00
Alexander Neumann
6dee59b789
Install gox before checking out code
...
Otherwise newer Go versions complain that the hash for the installed
version of gox is not in the go.mod, which we don't want anyways because
the tests should use the latest version of gox.
2022-03-20 11:26:56 +01:00
Alexander Neumann
2e19d19216
Use latest Go version for cross-compile and lint
2022-03-20 11:26:56 +01:00
Alexander Neumann
18a1de0de1
Use "go get" or "go install" selectively
...
Go 1.18 dropped support for installing binaries via "go get", Go <= 1.16
does not support it. So we need to use the right verb depending on the
Go version.
2022-03-20 11:26:56 +01:00
Alexander Neumann
9b57fcc6b0
Fix build.go, minimum Go version is 1.14
2022-03-20 10:54:33 +01:00
Alexander Neumann
17878036d8
Update tests to Go 1.18
2022-03-20 10:54:24 +01:00
Jason Lenz
2b1932a258
Report symlink sizes from FUSE mount for snapshot dir
...
Fixes #3667 .
2022-03-17 22:21:47 -05:00
greatroar
fdc738fb70
Report symlink sizes from FUSE mount
...
Fixes #3667 .
2022-03-13 16:48:35 +01:00
MichaelEischer
daea461f15
Merge pull request #3663 from jimt/find-msgs
...
Remove period from find messages
2022-03-07 22:23:49 +01:00
Jim Tittsler
a3d99217a4
Remove period from find messages
...
Simplifies cut-and-paste of IDs (and makes stylistically
consistent with other messages) #3659
2022-03-07 11:16:04 +09:00
MichaelEischer
e0ab689ccd
Merge pull request #3664 from DanielG/fix-doc-warning
...
doc: Fix block quote warning
2022-03-06 21:56:26 +01:00
Michael Eischer
7af69fd7b9
list: Never lock the repository when listing lock files
...
There's no point in locking the repository just to list the currently
existing lock files. This won't work for an exclusively locked
repository and is also confusing to users.
2022-03-06 21:44:51 +01:00
Daniel Gröber
49b67c8aaa
doc: Fix block quote warning
2022-03-06 18:15:55 +01:00
rawtaz
44d543ede3
Merge pull request #3653 from MichaelEischer/fix-ls-option-help
...
ls: Fix description for --host, --tag and --path options
2022-02-19 23:06:24 +01:00
Michael Eischer
5ef4ee7760
ls: Fix description for --host, --tag and --path options
2022-02-19 22:36:02 +01:00
Michael Eischer
254c8743fc
Limit number of large tree blobs loaded in parallel by StreamTrees
...
Load tree blobs with more than 50MB only from a single goroutine. Very
large tree blobs with for example 400 MB size can otherwise require
roughly 1GB * streamTreeParallelism memory.
2022-02-19 12:26:09 +01:00
MichaelEischer
ad4f4dbc7a
Merge pull request #3645 from greatroar/stdin-parent
...
Don't set a parent for --stdin backups
2022-02-19 11:36:51 +01:00
greatroar
63f6a9b085
Don't set a parent for --stdin backups
...
Loading any parent tree for these only wastes time and memory.
Fixes #3641 , where it was shown that the most recent tree will get
picked.
--parent is now implicitly ignored when --stdin is given.
2022-02-19 10:41:33 +01:00
MichaelEischer
4a2d5a146d
Merge pull request #3507 from ahmgithubahm/document-AWS_PROFILE-support
...
Document AWS_PROFILE support
2022-02-18 23:40:25 +01:00
Michael Eischer
1efc26899d
Update docs for AWS_PROFILE and AWS_SHARED_CREDENTIALS_FILE
2022-02-18 23:31:10 +01:00
MichaelEischer
8df246d0f3
Merge pull request #3628 from gum3ng/issue_3127
...
[#issue 3127] Add xattr support for Solaris
2022-02-17 22:07:39 +01:00
Michael Eischer
27524979e8
restorer: Remove dead code
2022-02-13 11:43:09 +01:00
gum3ng
dd30083c2b
[#issue 3127] Add xattr support for Solaris
2022-02-13 14:24:37 +05:30
MichaelEischer
fb4c5af5c4
Merge pull request #3642 from gco/master
...
Fix test failures on Solaris
2022-02-12 22:07:14 +01:00
MichaelEischer
18ec49ddfa
Merge pull request #3644 from duritong/centos-epel-repo
...
add a note about installation via epel for RHEL / CentOS Stream 8 & 9
2022-02-12 21:41:46 +01:00
Michael Eischer
5ec312ca06
sftp: Implement atomic uploads
...
Create a temporary file with a sufficiently random name to essentially
avoid any chance of conflicts. Once the upload has finished remove the
temporary suffix. Interrupted upload thus will be ignored by restic.
2022-02-12 20:30:49 +01:00
Michael Eischer
aebd24e414
Add changelog
2022-02-12 20:21:58 +01:00
Michael Eischer
d72181c8c1
Ensure that the lock cleanup handler is run after the global one
...
cleanup handlers run in the order in which they are added. As Go calls
init() functions in lexical order, the cleanup handler from global.go
was registered before that from lock.go, which is the correct order.
Make this order explicit to ensure that this won't break accidentally.
2022-02-12 20:21:58 +01:00
Michael Eischer
c6fd13425b
remember the refreshed lock file even if removal failed
...
This ensures that restic won't create lots of new lock files without
deleting them later on.
In some cases a Delete operation on a backend can return a "File does
not exist" error even though the Delete operation succeeded. This can
for example be caused by request retries. This caused restic to forget
about the new lock file and continue trying to remove the old (already
deleted) lock file.
2022-02-12 20:21:58 +01:00
Michael Eischer
bba8ba7a5b
repository: cancel streampack context after error
2022-02-12 20:18:25 +01:00
Michael Eischer
47554a3428
repository: Fix error handling in repack
...
When storing a blob fails, this is a fatal error which must not be
retried.
2022-02-12 20:18:25 +01:00
Michael Eischer
4b3dc415ef
checker: cleanup header extraction
2022-02-12 20:18:25 +01:00
Michael Eischer
930a00ad54
checker: reuse bufio reader
2022-02-12 20:18:25 +01:00
Michael Eischer
34ebafb8b6
repository: don't crash if blob size is too short
2022-02-12 20:18:25 +01:00
Michael Eischer
becebf5d88
repository: remove unused DownloadAndHash
2022-02-12 20:18:25 +01:00
Michael Eischer
f1e58e7c7f
checker: rewrite ReadData to stream packs
2022-02-12 20:18:25 +01:00
Michael Eischer
f40abd92fa
restorer: convert to use StreamPack
2022-02-12 20:18:25 +01:00
Michael Eischer
f00f690658
repository: stream packs during repacking
2022-02-12 20:18:25 +01:00
Michael Eischer
c4a2bfcb39
repository: Add StreamPacks function
...
The function supports efficiently loading a specified list of blobs from
a single pack in a streaming fashion. That is there's no need for
temporary files independent of the pack size.
2022-02-12 20:18:25 +01:00
Michael Eischer
153e2ba859
repository: Implement lisiting blobs per pack file
2022-02-12 20:18:24 +01:00
MichaelEischer
cc90f2ba6b
Merge pull request #2816 from greatroar/noatime
...
Set O_NOATIME flag on Linux
2022-02-07 21:38:31 +01:00
MichaelEischer
d8f58fb7bf
Merge pull request #3592 from vgt/jsondiff
...
Add json output for diff command
2022-02-07 21:33:30 +01:00
duritong
a4786dda5a
Update doc/020_installation.rst
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2022-02-06 21:31:18 +01:00
Michael Eischer
aaa7f94139
Add changelog for O_NOATIME
2022-02-06 15:00:37 +01:00
Michael Eischer
6b17a7110c
backup: Set O_NOATIME in the right place
...
The archiver uses FS.OpenFile, where FS is an instance of the FS
interface. This is different from fs.OpenFile, which uses the OpenFile
method provided by the fs package.
2022-02-06 15:00:37 +01:00
greatroar
7080fed7ae
Set O_NOATIME flag on Linux
...
Citing Kerrisk, The Linux Programming Interface:
The O_NOATIME flag is intended for use by indexing and backup
programs. Its use can significantly reduce the amount of disk
activity, because repeated disk seeks back and forth across the
disk are not required to read the contents of a file and to update
the last access time in the file’s i-node[.]
restic used to do this, but the functionality was removed along with the
fadvise call in #670 .
2022-02-06 15:00:34 +01:00
Michael Eischer
74f29ad09b
diff: add basic test for json output format
2022-02-06 11:44:15 +01:00
Michael Eischer
5f34ad523f
diff: fix test failure and add remark on quiet to changelog
2022-02-05 23:02:07 +01:00
MichaelEischer
58236ead12
Merge pull request #3619 from aneeshusa/avoid-time-travel-paradoxes-when-finding-parents
...
Avoid choosing parent snapshot newer than time of current snapshot
2022-02-05 22:51:24 +01:00
Michael Eischer
8ae4d86a84
rename snapshot timestamp filter variable
2022-02-05 22:42:38 +01:00
mh
3f0184ba2a
add a note about installation via epel
2022-02-05 22:18:33 +01:00
rawtaz
90473ea9ff
Merge pull request #3638 from leogott/patch-1
...
Documentation: Update msys2 wiki url after move
2022-01-27 17:07:45 +01:00
Leona 'leo' Gottfried
4e84e8ab3f
Update msys2 wiki url after move
...
https://github.com/msys2/msys2/wiki/Porting was permanently moved and redirects to https://www.msys2.org/
I substituted the new location of the /wiki/Porting page in the docs
2022-01-27 11:09:50 +01:00
Greg
2e9180638e
Fix test failures on Solaris
...
Add exceptions for symlinks, sticky bits, and device nodes in the same places where the BSDSs and/or Darwin have them.
2022-01-25 18:05:56 -08:00
Aneesh Agrawal
058dfc20da
Avoid choosing parent snapshot newer than time of current snapshot
...
Currently, `restic backup` (if a `--parent` is not provided)
will choose the most recent matching snapshot as the parent snapshot.
This makes sense in the usual case,
where we tag the snapshot-being-created with the current time.
However, this doesn't make sense if the user has passed `--time`
and is currently creating a snapshot older than the latest snapshot.
Instead, choose the most recent snapshot
which is not newer than the snapshot-being-created's timestamp,
to avoid any time travel.
Impetus for this change:
I'm using restic for the first time!
I have a number of existing BTRFS snapshots
I am backing up via restic to serve as my initial set of backups.
I initially `restic backup`'d the most recent snapshot to test,
then started backing up each of the other snapshots.
I noticed in `restic cat snapshot <id>` output
that all the remaining snapshots have the most recent as the parent.
2022-01-23 23:55:00 -05:00
Aneesh Agrawal
502fc3281c
Add CONTRIBUTING.md docs to not edit man pages
...
Document this code review feedback I got for other contributors.
2022-01-23 23:54:24 -05:00
David Vogt
77c850148a
Add json output for diff command
2022-01-23 23:22:26 +01:00
MichaelEischer
df89aa0087
Merge pull request #3623 from invine/issue-3464
...
Skip lock file creation on forget with --no-lock and --dry-run
2022-01-23 18:12:34 +01:00
Pavel Frolov
792523b28b
[issue 3464] skip lock creation in case of dry-run
2022-01-23 06:44:41 +03:00
MichaelEischer
f0a8182493
Merge pull request #3626 from fkusche/issue-3620-doc-unreferenced-packs
...
Update documentation regarding unreferenced packs
2022-01-22 15:15:25 +01:00
Florian Kusche
6183d0be53
Update output of restic check
2022-01-16 12:34:20 +01:00
rawtaz
7f6fc78f95
Merge pull request #3544 from restic/fix-b2-delete-retry
...
b2: Successful delete if file does not exist
2022-01-13 23:28:30 +01:00
rawtaz
abfbacf3d3
Merge pull request #3591 from MichaelEischer/prune-fix-max-repack
...
prune: Handle --max-repack-size=0 as expected
2022-01-13 03:53:20 +01:00
Florian Kusche
b0c1d0f9cd
Update documentation regarding unreferenced packs
...
Also removes an unnecessary space at the end of the last line.
2022-01-09 11:45:03 +01:00
rawtaz
8b6fe845d4
Merge pull request #3618 from mattxtaz/master
...
Add missing colon in prune stats output and realign the fields
2022-01-06 23:07:56 +01:00
mattxtaz
6ff32ee4d3
Add missing colon in prune stats output and change padding to 14 chars to align the fields
2022-01-06 21:15:15 +00:00
rawtaz
2ff3b7d69c
Merge pull request #3615 from gum3ng/issue_3558
...
doc: Add a FAQ section for invalid Windows filenames
2022-01-02 18:06:02 +01:00
gum3ng
9589de16db
[issue 3558]: Add a FAQ section for invalid Windows filenames
2022-01-02 22:24:00 +05:30
MichaelEischer
2c3e5d943d
Merge pull request #3593 from DarkKirb/parallelize-restic-copy
...
Parallelize blob upload/download for restic copy
2021-12-29 22:31:54 +01:00
Charlotte 🦝 Delenk
e2bb384a60
Parallelize blob upload/download for restic copy
...
Currently restic copy will copy each blob from every snapshot serially,
which has performance implications on high-latency backends such as b2.
This commit introduces 8x parallelism for blob downloads/uploads which
can improve restic copy operations up to 8x for repositories with many
small blobs on b2.
This commit also addresses the TODO comment in the copyTree function.
Related work:
A more thorough improvement of the restic copy performance can be found
in PR #3513
2021-12-29 18:59:09 +01:00
MichaelEischer
e5985e0d63
Merge pull request #3602 from cqjjjzr/fix-rclone-sigint
...
fix: rclone receiving SIGINT prematurely on Windows causing restic hang forever (#3601 )
2021-12-29 16:56:37 +01:00
MichaelEischer
8832837a8a
Merge pull request #3607 from jtagcat/quieter_cleanup
...
logging: quiet 'removing n old cache dirs'
2021-12-29 16:06:35 +01:00
jtagcat
f92130d878
logging: quiet 'removing n old cache dirs'
...
Closes #3595
Choosing to include `stdoutIsTerminal()` as:
- all other instances with `!opts.JSON` do so
- this likely will not affect anything, especially when autorun
- this seems to not be a meaningful enough summary
to include in auto-backup reports
JSON is still likely not guaranteed to work and this is a suboptimal
solution to this. Ideally, #1804 should refactor all print statements,
and define+document(+handle) when stdoutIsTerminal() should be used.
Else, it may end up more inconsistent and bulky
(duplicate lines, longer files).
2021-12-29 01:08:29 +02:00
Charlie Jiang
a5b0e0bef4
fix: rclone receiving SIGINT prematurely on Windows causing restic hang forever
...
Co-authored-by: greatroar <61184462+greatroar@users.noreply.github.com>
2021-12-28 13:14:46 +08:00
rawtaz
e6e51b84ac
Merge pull request #3605 from gum3ng/issue_3463
...
Improve clarity of error message in restic ls command
2021-12-27 21:45:23 +01:00
MichaelEischer
c5c3dfe10f
Merge pull request #3590 from metalsp0rk/documentation-enhance
...
Document Safe Passwords and Clarify B2 App Key information
2021-12-27 20:21:43 +01:00
Kyle Brennan
19ec4d8f17
Document safe passwords. Fix #2238
2021-12-27 10:43:18 -08:00
Kyle Brennan
47ecd950b8
Enhance details about user application keys. Fix #2672
2021-12-27 10:43:15 -08:00
MichaelEischer
051cc7ce71
Merge pull request #3589 from metalsp0rk/copy-no-lock
...
Make Copy respect no lock
2021-12-27 19:11:02 +01:00
Michael Eischer
64e733f3d6
tweak copy --no-lock changelog
2021-12-27 18:22:25 +01:00
Gautam Menghani
017614c41a
[#issue 3463] Improve clarity of error message in restic ls command
2021-12-27 22:42:27 +05:30
Michael Eischer
0cfdb82ea4
prune: Handle --max-repack-size=0 as expected
...
Previously the flag was ignored and `--max-repack-size=1` had to be
used.
2021-12-27 15:48:56 +01:00
Kyle Brennan
d5ed5da85c
update changelog for copy --no-lock
2021-12-03 12:16:40 -08:00
Kyle Brennan
8eb83029a8
Make copy honor --no-lock
2021-12-03 09:50:28 -08:00
MichaelEischer
882d58abce
Merge pull request #3163 from palbr/patch-1
...
Add PGP fingerprint to 020_installation.rst
2021-11-19 23:58:50 +01:00
Peter Albrecht
8de4401bb5
Changed URL for key-file
...
The keyfile provided by restic's own webserver (https://restic.net ) should be
more stable than relying on public keyservers. So I changed the URL to the
GPG keyfile, as recommended by MichaelEischer.
2021-11-19 15:47:59 +01:00
MichaelEischer
f7a9b90eb9
Merge pull request #3573 from magandrez/feat/ls-report-symbolic-notation
...
Provide mode in symbolic notation for `restic ls --json`
2021-11-18 21:29:04 +01:00
MichaelEischer
aa214f99b4
Merge pull request #3565 from mathstuf/doc-missing-metadata
...
doc: mention metadata that is not backed up currently
2021-11-18 21:28:55 +01:00
MichaelEischer
4a25bbaed3
Merge pull request #3578 from adsultana/patch-1
...
docs: Fix link to "help wanted" issues
2021-11-18 21:20:30 +01:00
Michael Eischer
583edc39b8
doc: reorder backup metadata exceptions
2021-11-18 21:17:38 +01:00
Manuel González
212b2f651f
Add file mode in symbolic notation to `ls --json`
...
This aligns `restic ls --json` with `restic find --json`, utilizing the same
naming.
2021-11-16 19:45:50 +02:00
Andrew Sultana
15ab96ecd6
docs: Fix link to "help wanted" issues
2021-11-16 00:03:34 +00:00
Ben Boeckel
d71afb3d32
doc: mention metadata that is not backed up currently
...
See: #3497
See: #1622
See: #2075
2021-11-13 18:40:26 -05:00
MichaelEischer
4bf05d91a1
Merge pull request #3571 from garrmcnu/blazer-unknown-authority
...
Update Backblaze B2 blazer module
2021-11-13 22:46:49 +01:00
MichaelEischer
de3afc1005
Merge pull request #3574 from gurjeet/rename_aws_s3_to_amazon_s3
...
Use S3's proper product name, Amazon S3
2021-11-13 22:30:32 +01:00
Michael Eischer
2ea998f70e
Add PR link to changelog
2021-11-13 22:25:54 +01:00
Garry McNulty
e8fa3855e7
Update blazer
...
If a request fails with "x509: certificate signed by unknown authority",
the B2 backend now returns the error without retrying the request.
Closes #3556
Closes #2355
2021-11-13 22:25:54 +01:00
Gurjeet Singh
34a6a24544
Use S3's proper product name, Amazon S3
...
Per Amazon's product page [1], S3 is officially called "Amazon S3". The
restic project uses the phrase "AWS S3" in some places. This patch
corrects the product name.
[1]:https://aws.amazon.com/s3/
2021-11-13 22:21:06 +01:00
MichaelEischer
1d8a0b06cb
Merge pull request #3575 from MichaelEischer/adjust-http2-canary-test
...
rest: Adjust http2-missing-eof test to golang >= 1.17.3
2021-11-13 22:13:24 +01:00
Michael Eischer
50053a85d3
rest: Adjust http2 missing eof test to golang >= 1.17.3, >= 1.16.10
...
The missing eof with http2 when a response included a content-length
header but no data, has been fixed in golang 1.17.3/1.16.10. Therefore
just drop the canary test and schedule it for removal once go 1.18 is
required as minimum version by restic.
2021-11-13 21:57:30 +01:00
MichaelEischer
f1cfb97237
Merge pull request #3514 from phcreery/rclone_timeout
...
rclone: extend timeout from 60s to 240s
2021-11-07 18:10:34 +01:00
MichaelEischer
cb81ee9396
Merge pull request #3474 from kitone/fix-issue-3382
...
Honor RESTIC_CACHE_DIR environment variable
2021-11-07 17:57:54 +01:00
Michael Eischer
b0e64deb27
rclone: Fix timeout calculation
2021-11-07 17:49:33 +01:00
phcreery
43d173b042
rclone: add timeout option and documentation
2021-11-07 17:49:21 +01:00
rawtaz
1b152a2c4d
Merge pull request #3568 from MichaelEischer/fix-local-fd-leak
...
local: Fix fd leak when encountering files directly inside data/
2021-11-07 01:36:48 +01:00
MichaelEischer
15cc3c0e23
Merge pull request #3566 from mathstuf/check-progress-output-stomping
...
check: wait for progress bar output
2021-11-06 23:34:49 +01:00
kitone
5904f80cfa
restic cache should display the name of the cache without shortening it in the case of the restic check
2021-11-06 20:18:51 +01:00
Ben Boeckel
4d579c4387
check: wait for progress bar output
...
Further code will also output to the terminal and the bar's cursor
positioning causes its output to overlap with the remaining output in a
racy way.
Fixes : #3344
2021-11-06 15:05:09 -04:00
Michael Eischer
15d7313387
local: Fix fd leak when encountering files directly inside data/
2021-11-06 19:44:57 +01:00
MichaelEischer
6c84ea1412
Merge pull request #3548 from gum3ng/issue_3490
...
Support for specifying amount of data in read-data-subset
2021-11-05 23:28:11 +01:00
MichaelEischer
78c7dd53ef
Merge pull request #3526 from greatroar/dump-refactor
...
Refactor internal/dump + concurrent load/write
2021-11-05 22:38:39 +01:00
MichaelEischer
a34bfa8269
Merge pull request #3562 from restic/rawtaz-mount-usage
...
mount: Improve usage information when mounted
2021-11-05 22:10:32 +01:00
kitone
0425a30420
add changelog issue-3382
2021-11-04 15:13:48 +01:00
kitone
1b23675f21
cache --cleanup should handle directories created by restic check.
...
Because there is no guarantee that a cleanup of these directories will occur
after the "restic check", we extend the behavior to detect and manage these
specific cache directories and allow their cleanup too.
2021-11-04 15:10:38 +01:00
Gautam Menghani
836fbb9133
[#issue 3490] Support for specifying file size in read-data-subset
2021-11-02 15:25:46 +05:30
greatroar
c71729dfc4
Refactor internal/dump + concurrent load/write
...
Package internal/dump has been reworked so its API consists of a single
type Dumper that handles tar and zip formats. Tree loading and node
writing happen concurrently.
2021-11-01 23:01:55 +01:00
Leo R. Lundgren
711ceb0109
mount: Improve usage information when mounted
2021-11-01 20:59:20 +01:00
MichaelEischer
829c0a67af
Merge pull request #3520 from nxt-engineering/Docker
...
Docker Multistage build
2021-10-24 17:37:54 +02:00
Alexander Neumann
fb5d9345a7
Merge pull request #3510 from MichaelEischer/fix-archiver-early-on-abort
...
archiver: Fix TestArchiverAbortEarlyOnError test
2021-10-16 15:37:41 +02:00
kitone
95eb859b54
Honor RESTIC_CACHE_DIR environment variable
...
Fix #3382 : restic check doesn't obey the RESTIC_CACHE_DIR environment variable
2021-10-10 16:00:02 +02:00
Michael Eischer
257740b0cc
b2: Successful delete if file does not exist
...
When deleting a file, B2 sometimes returns a "500 Service Unavailable"
error but nevertheless correctly deletes the file. Due to retries in
the B2 library blazer, we sometimes also see a "400 File not present"
error. The retries of restic for the delete request then fail with
"404 File with such name does not exist.".
As we have to rely on request retries in a distributed system to handle
temporary errors, also consider a delete request to be successful if the
file is reported as not existing. This should be safe as B2 claims to
provide a strongly consistent bucket listing and thus a missing file
shouldn't mysteriously show up again later on.
2021-10-09 23:51:12 +02:00
MichaelEischer
46d08d9404
Merge pull request #3535 from jtagcat/writingverbose
...
Change: selfupdate: 'writing restic to' as verbose
2021-10-09 22:41:51 +02:00
jtagcat
a7853057ab
Change: selfupdate: 'writing restic to' as verbose
...
Running restic self-update --quiet no longer
prints "writing restic to /usr/local/bin/restic".
The only output printed with -q is failures or
"successfully updated restic to version 0.12.1"
https://github.com/restic/restic/pull/3535
fix test fail: changelog title can't end with `.`
shorten changelog title
2021-10-09 23:24:18 +03:00
MichaelEischer
eb282532dc
Merge pull request #3534 from jtagcat/patch-1
...
docs/example: useradd: restic now system account
2021-10-09 20:42:43 +02:00
MichaelEischer
f2a3b3b4a1
Merge pull request #3537 from greatroar/dump-lru
...
Check cap instead of len in bloblru
2021-10-09 20:00:27 +02:00
MichaelEischer
58e8b34633
Merge pull request #3532 from garrmcnu/s3-credentials-config
...
s3: Add warning if key ID or secret is empty
2021-10-09 19:32:32 +02:00
MichaelEischer
a02cea6e83
Merge pull request #3539 from jnoxon/fix-ec2-metadata-v2
...
get iam roles working with ec2 metadata v2
2021-10-07 23:00:27 +02:00
Garry McNulty
708d7a2574
s3: Add warning if key ID or secret is empty
...
Also add debug message if no credential types are available.
Closes #2388
2021-10-06 23:13:40 +01:00
Jeff Noxon
6f4b5ab8d1
get iam roles working with ec2 metadata v2
2021-10-03 20:06:55 -05:00
greatroar
634a9c162d
Check cap instead of len in bloblru
...
restic dump uses bloblru.Cache to keep buffers alive, but also reuses
evicted buffers. That means large buffers may be used to store small
blobs, causing the cache to think it's using less memory than it
actually does.
2021-10-03 09:34:17 +02:00
jtagcat
632ca2ef52
docs/example: useradd: restic now system account
...
and: use /sbin/nologin
and: use long flags in the useradd command
and: download v0.12.1 instead of v0.9.6
2021-10-03 00:51:06 +03:00
MichaelEischer
24088f8307
Merge pull request #3528 from greatroar/tree-insert
...
internal/restic: Don't allocate in Tree.Insert
2021-09-27 22:22:03 +02:00
greatroar
c892c0bab9
internal/restic: Don't allocate in Tree.Insert
...
name old time/op new time/op delta
BuildTree-8 34.6µs ± 4% 7.0µs ± 3% -79.68% (p=0.000 n=18+19)
name old alloc/op new alloc/op delta
BuildTree-8 34.0kB ± 0% 0.9kB ± 0% -97.37% (p=0.000 n=20+20)
name old allocs/op new allocs/op delta
BuildTree-8 108 ± 0% 1 ± 0% -99.07% (p=0.000 n=20+20)
2021-09-26 18:08:48 +02:00
MichaelEischer
78dac2fd48
Merge pull request #3523 from greatroar/swift-v2
...
Upgrade ncw/swift to v2
2021-09-24 21:41:36 +02:00
Michael Eischer
5ea8bba1a1
swift: restore context err check for list()
2021-09-24 21:19:46 +02:00
MichaelEischer
a5e103a212
Merge pull request #3522 from greatroar/dump-lru
...
Use LRU cache in restic dump
2021-09-24 20:33:58 +02:00
greatroar
e7ec0453b1
Upgrade ncw/swift to v2
2021-09-24 19:08:37 +02:00
Uli Martens
1ebcb1d097
Add changelog entry for PR #3508
2021-09-24 15:45:09 +02:00
greatroar
fe04d024c7
Use LRU cache in restic dump
2021-09-24 15:45:08 +02:00
Uli Martens
718966a81a
Move Blobcache into dedicated internal package
2021-09-24 15:45:00 +02:00
greatroar
4f33eca634
Remove unused Writer arg to internal/dump.writeDump
2021-09-24 15:40:42 +02:00
MichaelEischer
cc110c42e6
Merge pull request #2657 from mansam/add-skip-tls-verification-flag
...
Add --insecure-tls flag to disable SSL cert verification
2021-09-22 21:40:01 +02:00
Sam Lucidi
897d8e662c
Add --insecure-tls flag to disable SSL cert verification
...
Signed-off-by: Sam Lucidi <slucidi@redhat.com>
2021-09-21 10:52:40 -04:00
cimnine
4a95af5290
GitHub action for Docker build
2021-09-21 08:23:33 +02:00
cimnine
f28c8bc1c2
Multistage Docker build
2021-09-21 08:23:33 +02:00
MichaelEischer
1827b16ade
Merge pull request #3519 from greatroar/maphash
...
Replace siphash by hash/maphash
2021-09-19 19:46:03 +02:00
greatroar
8b758c78a3
Require Go 1.14 to build
2021-09-19 16:18:19 +02:00
greatroar
8d2996eaaa
Replace siphash by hash/maphash
...
In Go 1.17.1, maphash has become quite a bit faster than siphash, so we
can drop one third-party dependency. maphash is just an interface to the
standard Go map's hash function, which we already trust for other use
cases.
Benchmark results on linux/amd64, -benchtime=3s:
name old time/op new time/op delta
IndexHasUnknown-8 50.6ns ±10% 41.0ns ±19% -18.92% (p=0.000 n=9+10)
IndexHasKnown-8 52.6ns ±12% 41.5ns ±12% -21.13% (p=0.000 n=9+10)
IndexMapHash-8 3.64µs ± 1% 2.00µs ± 0% -45.09% (p=0.000 n=10+9)
IndexAlloc-8 700ms ± 1% 601ms ± 6% -14.18% (p=0.000 n=8+10)
IndexAllocParallel-8 205ms ± 5% 192ms ± 8% -6.18% (p=0.043 n=10+10)
MasterIndexAlloc-8 319ms ± 1% 279ms ± 5% -12.58% (p=0.000 n=10+10)
MasterIndexLookupSingleIndex-8 156ns ± 8% 147ns ± 6% -5.46% (p=0.023 n=10+10)
MasterIndexLookupMultipleIndex-8 150ns ± 7% 142ns ± 8% -5.69% (p=0.007 n=10+10)
MasterIndexLookupSingleIndexUnknown-8 74.4ns ± 6% 72.0ns ± 9% ~ (p=0.175 n=10+9)
MasterIndexLookupMultipleIndexUnknown-8 67.4ns ± 9% 65.5ns ± 7% ~ (p=0.340 n=9+9)
MasterIndexLookupParallel/known,indices=25-8 461ns ± 2% 445ns ± 2% -3.49% (p=0.000 n=10+10)
MasterIndexLookupParallel/unknown,indices=25-8 408ns ±11% 378ns ± 5% -7.22% (p=0.035 n=10+9)
MasterIndexLookupParallel/known,indices=50-8 479ns ± 1% 437ns ± 4% -8.82% (p=0.000 n=10+10)
MasterIndexLookupParallel/unknown,indices=50-8 406ns ± 8% 343ns ±15% -15.44% (p=0.001 n=10+10)
MasterIndexLookupParallel/known,indices=100-8 480ns ± 1% 455ns ± 5% -5.15% (p=0.000 n=8+10)
MasterIndexLookupParallel/unknown,indices=100-8 391ns ±18% 382ns ± 8% ~ (p=0.315 n=10+10)
MasterIndexLookupBlobSize-8 71.0ns ± 8% 57.2ns ±11% -19.36% (p=0.000 n=9+10)
PackerManager-8 254ms ± 1% 254ms ± 1% ~ (p=0.285 n=15+15)
name old speed new speed delta
IndexMapHash-8 1.12GB/s ± 1% 2.05GB/s ± 0% +82.13% (p=0.000 n=10+9)
PackerManager-8 208MB/s ± 1% 207MB/s ± 1% ~ (p=0.281 n=15+15)
name old alloc/op new alloc/op delta
IndexMapHash-8 0.00B 0.00B ~ (all equal)
IndexAlloc-8 400MB ± 0% 400MB ± 0% ~ (p=1.000 n=9+10)
IndexAllocParallel-8 401MB ± 0% 401MB ± 0% +0.00% (p=0.000 n=10+10)
MasterIndexAlloc-8 258MB ± 0% 262MB ± 0% +1.42% (p=0.000 n=9+10)
PackerManager-8 73.1kB ± 0% 73.1kB ± 0% ~ (p=0.382 n=13+13)
name old allocs/op new allocs/op delta
IndexMapHash-8 0.00 0.00 ~ (all equal)
IndexAlloc-8 907k ± 0% 907k ± 0% -0.00% (p=0.000 n=10+10)
IndexAllocParallel-8 907k ± 0% 907k ± 0% +0.00% (p=0.009 n=10+10)
MasterIndexAlloc-8 327k ± 0% 317k ± 0% -3.06% (p=0.000 n=10+10)
PackerManager-8 744 ± 0% 744 ± 0% ~ (all equal)
2021-09-19 16:05:18 +02:00
MichaelEischer
58efe21eca
Merge pull request #3264 from amozoss/upstream-master
...
Refactor backup progress
2021-09-19 14:54:42 +02:00
MichaelEischer
71fcf48533
Merge pull request #2594 from greatroar/concurrent-restore-verify
...
Concurrent restore --verify
2021-09-19 14:52:31 +02:00
Michael Eischer
921e328b56
restore: Fix linting error
2021-09-19 14:41:07 +02:00
Michael Eischer
e62d4f622f
Reword changelog
2021-09-19 14:41:07 +02:00
Michael Eischer
2cdc0719af
restorer: Sanitize verify errors
2021-09-19 14:01:26 +02:00
greatroar
bdcdfaf6b4
restore --verify: buffer reuse consistency and comment
2021-09-19 13:11:27 +02:00
greatroar
2b94742ca5
Replace no-op closures in restorer by nil check
2021-09-19 13:11:16 +02:00
greatroar
d357744104
Handle canceled context in restore --verify properly
2021-09-19 13:11:05 +02:00
greatroar
d4225ec803
Simplify buffer growing in Restorer.verifyFile
...
Suggested-by: Igor Fedorenko <igor@ifedorenko.com>
2021-09-19 13:10:55 +02:00
greatroar
de8521ae56
Report number of successes from Restorer.VerifyFiles
...
Previously, the number of attempts was reported.
2021-09-19 13:10:44 +02:00
greatroar
bb066cf7d3
Concurrent Restorer.VerifyFiles
...
Time to verify a 2GB snapshot down from 9.726s to 4.645s (-52%).
2021-09-19 13:10:41 +02:00
greatroar
556424d61b
Reuse buffer in Restorer.VerifyFiles
...
Time to verify a 2GB snapshot down from 11.568s to 9.726s (-16%).
2021-09-19 12:55:31 +02:00
greatroar
92ae951ffa
Report timing from restic restore --verify
2021-09-19 12:53:09 +02:00
greatroar
973fa921cb
Test and document Restorer.VerifyFiles
2021-09-19 12:52:11 +02:00
Michael Eischer
e0d615c264
archiver: Fix TestArchiverAbortEarlyOnError test
...
This can be caused when the test has uploaded four blobs, then queues
two blobs for upload which are delayed. Then a seventh file can be
opened which lead to a test failure.
2021-09-12 22:17:17 +02:00
MichaelEischer
ef5672a902
Merge pull request #3509 from ajaspers/patch-1
...
Update instructions for reproducing build
2021-09-12 21:40:37 +02:00
ajaspers
c0eddc9969
Update instructions for reproducing build
...
Dependencies are fetched at build time and stored in the GOPATH. These paths end up being in the final binary.
Bump restic version to latest and go version to the 1.16.6, which was used to build restic 0.12.1.
2021-09-12 09:02:57 -07:00
Michael Eischer
fbb0e6499a
ui: consolidate backup ui in ui/backup package
2021-09-12 16:20:15 +02:00
Michael Eischer
503d4c3e2f
Add changelog
2021-09-12 16:00:49 +02:00
Michael Eischer
cccb0d4064
ui: assert that ProgressPrinter interface is implemented
2021-09-12 15:26:40 +02:00
Michael Eischer
a144c986f2
backup: Reenable JSON status updates with redirected output
...
After the refactoring status updates were no longer printed in quiet
mode or when the output is not an interactive terminal. However, the
JSON output is often piped to e.g. another program. Thus, don't set the
update frequency to 0 in that case. The status updates are still
disabled for backup --quiet.
This also reduces the status update frequency to 60fps compared to a
potentially much higher value before the refactoring.
2021-09-12 15:26:40 +02:00
Michael Eischer
d62bfed65d
ui: move SetDryRun to ProgressReporter
2021-09-12 15:25:58 +02:00
Michael Eischer
77b129ec74
ui: remove unused Summary field
2021-09-12 15:25:58 +02:00
Michael Eischer
3024239e40
ui/json: restore summary output
2021-09-12 15:25:58 +02:00
Michael Eischer
5ccf583b8a
ui: restore a few comments
2021-09-12 15:25:58 +02:00
Michael Eischer
80cbaf6d38
ui: Simplify ReportTotal methods
2021-09-12 15:25:58 +02:00
Dan Willoughby
448419990c
Refactor backup progress
...
Move the shared logic into the progress
Allows logic to be shared with forth coming restore status
2021-09-12 15:25:58 +02:00
Andy
7baa9a570d
document AWS_PROFILE support
...
Since restic (or whatever library it is using) seems to respect/use AWS_PROFILE, it's worth documenting this.
2021-09-10 14:06:46 +01:00
Alexander Neumann
bf9c8771a4
Merge pull request #3482 from MichaelEischer/changelog-3429
...
Add changelog for #3246 and #3429
2021-09-08 09:15:40 +02:00
Michael Eischer
5e84f38f31
Add changelog for #3246
2021-09-07 21:18:11 +02:00
Alexander Neumann
8fe122d675
Merge pull request #3488 from MichaelEischer/rebuild-broken-index
...
Fix `rebuild-index` for damaged index
2021-09-07 17:00:23 +02:00
Alexander Neumann
74c47f1f12
Merge pull request #3502 from restic/rawtaz-issue-519
...
doc: Add note about scheduling to backup section
2021-09-07 16:58:17 +02:00
MichaelEischer
fa5ca8af81
Merge pull request #2856 from aawsome/remove-readahead
...
Simplify cache logic
2021-09-04 20:25:49 +02:00
MichaelEischer
b45d88e124
Merge pull request #3496 from juergenhoetzel/imporove-snapshot-filter-warning-message
...
Improve snapshot filter warning message
2021-09-03 23:42:27 +02:00
MichaelEischer
bc4cbd775b
Merge pull request #2880 from aawsome/enhance-recover
...
Improve recover command
2021-09-03 23:40:43 +02:00
MichaelEischer
a29777f467
Merge pull request #3501 from greatroar/printprogress
...
Streamline progress printing in cmd/restic
2021-09-03 23:34:36 +02:00
Alexander Weiss
bce87922c0
Improve recover
...
- only save directories not referenced by any snapshot
- dont't write empty snapshot
- use progress bar
2021-09-03 21:36:57 +02:00
Alexander Weiss
81876d5c1b
Simplify cache logic
2021-09-03 21:01:00 +02:00
greatroar
7f0aa49f45
cmd/restic: Streamline progress printing
...
* PrintProgress no longer does unnecessary Sprintf calls, and performs
fewer allocations in general
* newProgressMax's callback checks whether the terminal supports
line updates once instead of once per call
* the callback looks up the terminal width once per call instead of
twice (on Windows)
* the status shortening now uses the Unicode-aware version from
internal/ui/termstatus (future-proofing)
2021-09-03 11:48:22 +02:00
greatroar
5aaa3e93c1
internal/ui/termstatus: Optimize and publish Truncate
...
name old time/op new time/op delta
TruncateASCII-8 347ns ± 1% 69ns ± 1% -80.02% (p=0.000 n=9+10)
TruncateUnicode-8 447ns ± 3% 348ns ± 1% -22.04% (p=0.000 n=10+10)
2021-09-03 11:48:22 +02:00
MichaelEischer
ec2e3b260e
Merge pull request #3499 from greatroar/wrappedconn-pointer
...
rclone: Return one fewer value from run
2021-08-31 21:48:36 +02:00
rawtaz
26914abe62
doc: Add note about scheduling to backup section
...
Explains that restic doesn't have built-in scheduling
and mentions a few keywords one can search for.
2021-08-29 22:03:22 +02:00
greatroar
950b818274
rclone: Return one fewer value from run
2021-08-26 18:12:08 +02:00
Juergen Hoetzel
defe19fdf6
Quote snapshot prefix in error string
2021-08-25 16:11:28 +02:00
Juergen Hoetzel
409e4936af
Improve snapshot filter warning message
...
Include the root-cause why the snapshot prefix is ignored.
2021-08-25 15:46:21 +02:00
MichaelEischer
10b39d7591
Merge pull request #3487 from andreaso/rest-proto-readme-link
...
Fix REST protocol link in README file
2021-08-22 18:47:32 +02:00
Michael Eischer
194ed19557
Add changelog
2021-08-22 18:29:58 +02:00
Michael Eischer
877fc9f352
rebuild-index: test that invalid indexes are skipped and removed
2021-08-22 18:24:19 +02:00
Michael Eischer
64258a2c2a
rebuild-index: Ignore broken index files
...
Previously, an e.g. truncated index could prevent rebuild-index from
working.
2021-08-22 18:23:47 +02:00
Andreas Olsson
c520672982
Fix REST protocol link in README file
...
1. All other document links goes to the _Read the Docs_ site
2. The GitHub reStructuredText renderer doesn't appear to do includes,
making for a rather empty read.
2021-08-22 13:16:51 +02:00
Alexander Neumann
9374c3ce81
Merge pull request #3485 from lbausch/fix_typo
...
Fix typo in changelog
2021-08-21 13:16:20 +02:00
Lorenz Bausch
4d56b34096
Fix typo in changelog
2021-08-21 12:16:04 +02:00
Alexander Neumann
66382b2861
Update Go to 1.17
2021-08-17 21:38:34 +02:00
Michael Eischer
1fab5892b5
Add changelog for #3429
2021-08-16 17:42:16 +02:00
MichaelEischer
c898f7a6bf
Merge pull request #3479 from BUFU1610/patch-1
...
fixed --keep-within-X options in example
2021-08-15 20:06:26 +02:00
BUFU
7659790923
fixed --keep-within-X options in example
...
The order of the words was wrong in the example, i.e. fixed to --keep-within-hourly from --keep-hourly-within.
2021-08-15 19:56:22 +02:00
MichaelEischer
ecf34783ef
Merge pull request #3480 from MichaelEischer/fix-rest-tests
...
Fix rest backend tests
2021-08-15 18:59:02 +02:00
Michael Eischer
68370feeee
backends: Remove TestSaveFilenames test
...
Filenames are expected to match the sha256 sum of the file content. This
rule is now enforced by the rest server thus making this test useless.
2021-08-15 18:24:16 +02:00
Michael Eischer
574c83e47f
rest: Fix test to use paths which are the sha256 sum of the data
2021-08-15 18:19:43 +02:00
Michael Eischer
e6a5801155
rest: Fix test backend url
...
The rest config normally uses prepareURL to sanitize URLs and ensures
that the URL ends with a slash. However, the test used an URL without a
trailing slash, which after the rest server changes causes test
failures.
2021-08-15 18:16:17 +02:00
Alexander Neumann
d90efd7704
Fix test
2021-08-09 10:30:10 +02:00
Alexander Neumann
9fe5a87785
Merge pull request #3429 from MichaelEischer/safe-key-change
...
key: Check that new key works before deleting the old one
2021-08-09 10:07:15 +02:00
Alexander Neumann
7f1608dc77
Merge pull request #3246 from restic/content-hash-for-upload
...
Calculate content hashes during upload
2021-08-08 17:24:08 +02:00
Michael Eischer
f4c5dec05d
backend: test that a wrong hash fails an upload
2021-08-04 22:17:46 +02:00
Michael Eischer
7c1903e1ee
panic if hash returns an error
...
Add a sanity check that the interface contract is honoured.
2021-08-04 22:17:46 +02:00
Michael Eischer
51b7e3119b
mem: calculate md5 content hash for uploads
...
The mem backend is primarily used for testing. This ensures that the
upload hash calculation gets appropriate test coverage.
2021-08-04 22:17:46 +02:00
Michael Eischer
a009b39e4c
gs/swift: calculate md5 content hash for upload
2021-08-04 22:17:46 +02:00
Michael Eischer
1d3e99f475
azure: check upload using md5 content hash
...
For files below 256MB this uses the md5 hash calculated while assembling
the pack file. For larger files the hash for each 100MB part is
calculated on the fly. That hash is also reused as temporary filename.
As restic only uploads encrypted data which includes among others a
random initialization vector, the file hash shouldn't be susceptible to
md5 collision attacks (even though the algorithm is broken).
2021-08-04 22:17:46 +02:00
Michael Eischer
9aa2eff384
Add plumbing to calculate backend specific file hash for upload
...
This enables the backends to request the calculation of a
backend-specific hash. For the currently supported backends this will
always be MD5. The hash calculation happens as early as possible, for
pack files this is during assembly of the pack file. That way the hash
would even capture corruptions of the temporary pack file on disk.
2021-08-04 22:17:46 +02:00
Michael Eischer
ee2f14eaf0
s3: enable content hash calculation for uploads
2021-08-04 22:12:12 +02:00
MichaelEischer
553ea36ca6
Merge pull request #2838 from greatroar/cache-conflicts
...
Make cache concurrency-safe
2021-08-04 22:11:50 +02:00
greatroar
6586e90acf
Modernize internal/cache error handling
2021-08-04 22:02:42 +02:00
greatroar
ea04f40eb3
Save cached files to a temporary location first
2021-08-04 22:02:42 +02:00
greatroar
f9b6f8fd45
Replace duplicate type checking in cache with a function
2021-08-04 22:02:42 +02:00
MichaelEischer
1b1a2115fa
Merge pull request #3436 from greatroar/local-save-tmp
...
Save files under temporary name in local backend
2021-08-04 22:01:53 +02:00
Michael Eischer
65908647e3
Add changelog
2021-08-04 21:51:53 +02:00
greatroar
81e2499d19
Sync directory to get durable writes in local backend
2021-08-04 21:51:53 +02:00
greatroar
195a5cf996
Save files under temporary name in local backend
...
Fixes #3435 .
2021-08-04 21:51:53 +02:00
MichaelEischer
bc97a3d1f9
Merge pull request #3300 from aawsome/backup-dryrun
...
backup: add --dry-run/-n flag
2021-08-04 21:50:38 +02:00
Michael Eischer
702cff636f
Add use case to changelog
2021-08-04 21:19:29 +02:00
Alexander Weiss
780e11b7e2
Adapt changelog
2021-08-04 21:19:29 +02:00
erin
4126435663
resolve rawtaz's review comments
...
make majority of suggestions from review by @rawtaz verbatim, with one clarification on my part in changelog
2021-08-04 21:19:29 +02:00
Alexander Weiss
d107a2cfdf
Separate dry run tests
2021-08-04 21:19:29 +02:00
Alexander Weiss
38a8a48a25
Simplify dry run backend
2021-08-04 21:19:29 +02:00
Ryan Hitchman
77bf148460
backup: add --dry-run/-n flag to show what would happen.
...
This can be used to check how large a backup is or validate exclusions.
It does not actually write any data to the underlying backend. This is
implemented as a simple overlay backend that accepts writes without
forwarding them, passes through reads, and generally does the minimal
necessary to pretend that progress is actually happening.
Fixes #1542
Example usage:
$ restic -vv --dry-run . | grep add
new /changelog/unreleased/issue-1542, saved in 0.000s (350 B added)
modified /cmd/restic/cmd_backup.go, saved in 0.000s (16.543 KiB added)
modified /cmd/restic/global.go, saved in 0.000s (0 B added)
new /internal/backend/dry/dry_backend_test.go, saved in 0.000s (3.866 KiB added)
new /internal/backend/dry/dry_backend.go, saved in 0.000s (3.744 KiB added)
modified /internal/backend/test/tests.go, saved in 0.000s (0 B added)
modified /internal/repository/repository.go, saved in 0.000s (20.707 KiB added)
modified /internal/ui/backup.go, saved in 0.000s (9.110 KiB added)
modified /internal/ui/jsonstatus/status.go, saved in 0.001s (11.055 KiB added)
modified /restic, saved in 0.131s (25.542 MiB added)
Would add to the repo: 25.892 MiB
2021-08-04 21:19:29 +02:00
MichaelEischer
533ac4fd95
Merge pull request #3467 from greatroar/wrappedconn-pointer
...
Use rclone.wrappedConn by pointer
2021-08-04 21:18:53 +02:00
Alexander Neumann
7049f1cbfc
Set development version for 0.12.1
2021-08-03 11:45:39 +02:00
Alexander Neumann
dc7a8aab24
Add version for 0.12.1
2021-08-03 11:45:36 +02:00
Alexander Neumann
94983a1f36
Update manpages and auto-completion
2021-08-03 11:45:36 +02:00
Alexander Neumann
a92faca10e
Generate CHANGELOG.md for 0.12.1
2021-08-03 11:45:19 +02:00
Alexander Neumann
b19cd8c50f
Prepare changelog for 0.12.1
2021-08-03 11:45:19 +02:00
Alexander Neumann
b862732318
Merge pull request #3468 from restic/rework-changelog
...
Reword changelog entries
2021-08-03 10:06:56 +02:00
Leo R. Lundgren
cb844e7136
Polish changelog entries
2021-08-03 00:01:09 +02:00
Alexander Neumann
b7fe1fe6b4
Reword changelog entries
2021-08-03 00:01:09 +02:00
MichaelEischer
c98bbdcdbe
Merge pull request #3457 from systemmonkey42/feature/untagged
...
Feature to match untagged snapshots only when listing or forgetting
2021-08-02 23:19:39 +02:00
David le Blanc
326fefcd80
Allow --tag and --keep-tag to match untagged snapshots
2021-08-02 23:06:20 +02:00
greatroar
fa3eed1998
Use rclone.wrappedConn by pointer
...
This shaves a kilobyte off the Linux binary by not generating a
non-pointer interface implementation.
2021-08-01 09:11:50 +02:00
MichaelEischer
5571c3f7fd
Merge pull request #3453 from MichaelEischer/http2-zero-length-workaround
...
rest: Workaround Http2 zero-length reply bug
2021-07-31 20:30:06 +02:00
Alexander Neumann
d8ea10db8c
rest: Rework handling HTTP2 zero-length replies bug
...
Add comment that the check is based on the stdlib HTTP2 client. Refactor
the checks into a function. Return an error if the value in the
Content-Length header cannot be parsed.
2021-07-31 17:12:24 +02:00
Alexander Neumann
77551597b2
Merge pull request #3416 from torfason/keep-hourly-within
...
Keep hourly within
2021-07-30 10:36:41 +02:00
Alexander Neumann
92f293cd0b
Merge pull request #3427 from MichaelEischer/find-packs-from-index
...
find: List missing pack files based on the index
2021-07-30 10:33:02 +02:00
Magnus Thor Torfason
2081bd12fb
forget: Ensure future snapshots do not affect --keep-within-*
...
Ensure that only snapshots made in the past are taken into account when running restic forget with the within switches (--keep-within, --keep-within- hourly, and friends)
2021-07-24 16:14:43 +00:00
Magnus Thor Torfason
74ebc650ab
forget: Add --keep-within-hourly (and friends)
...
Allow keeping hourly/daily/weekly/monthly/yearly snapshots for a given time period.
This adds the following flags/parameters to restic forget:
--keep-within-hourly duration
--keep-within-daily duration
--keep-within-weekly duration
--keep-within-monthly duration
--keep-within-yearly duration
Includes following changes:
- Add tests for --keep-within-hourly (and friends)
- Add documentation for --keep-within-hourly (and friends)
- Add changelog for --keep-within-hourly (and friends)
2021-07-24 16:14:43 +00:00
Alexander Neumann
c707d71b72
Merge pull request #3401 from MichaelEischer/goroutine-shutdown-cleanups
...
Goroutine shutdown cleanups
2021-07-11 16:32:28 +02:00
Alexander Neumann
691866ce43
Merge pull request #3452 from MichaelEischer/add-s390x-releases
...
Add release binaries for linux/s390x
2021-07-11 16:30:25 +02:00
Alexander Neumann
efd918c59e
Merge pull request #3454 from MichaelEischer/update-dependencies
...
Update dependencies
2021-07-11 16:29:31 +02:00
Michael Eischer
7d28006e2e
Add changelog
2021-07-10 22:39:01 +02:00
Michael Eischer
0880afe67b
Use our generate command instead of cobra's complete command
2021-07-10 19:44:18 +02:00
Michael Eischer
100baf74c0
Update cobra
...
The most noteworthy change seems to be
https://github.com/spf13/cobra/pull/1192
Have Cobra create 'completion' command automatically
2021-07-10 19:43:14 +02:00
Michael Eischer
c733ae6b16
Readd and update indirect dependencies
...
The azure-sdk-for-go is the only remaining module without a go.mod file.
Thus we only need indirect dependencies for it. Remove all other
indirect dependencies.
2021-07-10 19:21:43 +02:00
Michael Eischer
989b398fee
Misc library updates
...
The most noteworthy change is the xattr update which includes
https://github.com/pkg/xattr/pull/54 that adds xattr support for solaris
and illumos
2021-07-10 19:04:47 +02:00
Michael Eischer
bbc8146934
Update minio/sha256-simd
...
Apparently the standard Go sha256 implementation is now faster than the
assembly implementation. The library now only adds support for SHA
extensions available in some processors.
See https://github.com/minio/sha256-simd/pull/57 for more details.
2021-07-10 18:34:16 +02:00
Michael Eischer
aa22ebac69
Update backend dependencies
...
Possibly interesting changes:
https://github.com/Azure/azure-sdk-for-go/pull/14521
Retry on HTTP client errors
https://github.com/minio/minio-go/pull/1452
fix: make sure getObject returns error on truncated responses
f5854403a9
http2: close Transport connection on write errors
2021-07-10 18:27:13 +02:00
Michael Eischer
097ed659b2
rest: test that zero-length replies over HTTP2 work correctly
...
The first test function ensures that the workaround works as expected.
And the second test function is intended to fail as soon as the issue
has been fixed in golang to allow us to eventually remove the
workaround.
2021-07-10 17:22:42 +02:00
Michael Eischer
185a55026b
rest: workaround for HTTP2 zero-length replies bug
...
The golang http client does not return an error when a HTTP2 reply
includes a non-zero content length but does not return any data at all.
This scenario can occur e.g. when using rclone when a file stored in a
backend seems to be accessible but then fails to download.
2021-07-10 16:59:01 +02:00
Michael Eischer
495831d53c
add release binaries for linux/s390x
2021-07-10 00:10:24 +02:00
Michael Eischer
3442dc87fb
find: Address review comments
2021-07-06 21:04:34 +02:00
Michael Eischer
a81f34ae47
Add changelog
2021-07-06 21:04:34 +02:00
Michael Eischer
95b44490a0
find: search blob ids for pack in index if pack is missing
...
If a pack file is missing try to determine the contained pack ids based
on the repository index. This helps with assessing the damage to a
repository before running `rebuild-index`.
2021-07-06 21:04:34 +02:00
Michael Eischer
3caab3c7ac
find: Print not found pack files
2021-07-06 21:04:34 +02:00
Michael Eischer
40745b4f82
find: stop file listing after resolving all pack files
2021-07-06 21:04:34 +02:00
Michael Eischer
6c01078f3d
find: support resolving multiple pack ids to blobs
...
Just passing the list of blobs to packsToBlobs would also work in most
cases, however, it could cause unexpected results when multiple pack
files have the same prefix. Forget found prefixes to prevent this.
2021-07-06 21:04:34 +02:00
Leo R. Lundgren
790294dc26
contributing: Change freenode to libera
2021-07-04 11:05:36 +02:00
Alexander Neumann
30d968b0e4
Merge pull request #3449 from MichaelEischer/fix-restore-retries
...
restore: Correctly handle partial pack download errors
2021-07-04 10:55:09 +02:00
Michael Eischer
43b82d69b4
Add changelog
2021-06-29 21:27:00 +02:00
Michael Eischer
bd316d3893
restore: Test partial pack downloads in filerestorer
2021-06-29 21:11:30 +02:00
Michael Eischer
e8bbb05328
restore: Correctly handle partial pack download errors
...
Failed pack/blob downloads should be retried. For blobs that fail
decryption assume that the pack file is really damaged and try to
restore the remaining blobs.
2021-06-29 20:54:16 +02:00
rawtaz
58be5172ff
Merge pull request #3437 from MichaelEischer/fix-prune-output
...
prune: Add missing newlines in error descriptions
2021-06-28 00:56:51 +02:00
MichaelEischer
cb6fd281a0
Merge pull request #3442 from restic/rawtaz-doc-verbose
...
doc: Correct position of --verbose in backup docs
2021-06-27 18:19:37 +02:00
rawtaz
eb61de7b3a
Merge pull request #3287 from tjrana/clean-up-pr-template
...
Clean up PR template
2021-06-25 21:06:50 +02:00
rawtaz
98a88b483d
doc: Correct position of --verbose in backup docs
2021-06-25 19:26:46 +02:00
MichaelEischer
6a4c1ed50d
Merge pull request #3438 from MichaelEischer/fast-fuse-changelog
...
Add changelog for #3426
2021-06-20 15:05:19 +02:00
Michael Eischer
409306db2b
Add changelog for #3426
2021-06-20 14:46:37 +02:00
Michael Eischer
aad8864835
prune: Add missing newlines in error descriptions
2021-06-20 14:25:40 +02:00
MichaelEischer
c1eb7ac1a1
Merge pull request #3420 from greatroar/local-errs
...
Modernize error handling in local backend
2021-06-20 14:20:40 +02:00
greatroar
e5f0f67ba0
Modernize error handling in local backend
...
* Stop prepending the operation name: it's already part of os.PathError,
leading to repetitive errors like "Chmod: chmod /foo/bar: operation not
permitted".
* Use errors.Is to check for specific errors.
2021-06-18 11:13:27 +02:00
Alexander Neumann
45eb30388f
Merge pull request #3426 from MichaelEischer/fast-fuse
...
mount: enable fuse readahead
2021-06-14 10:27:41 +02:00
Michael Eischer
454b6d608e
key: Add test that failed key saves don't break the repository
2021-06-13 13:46:48 +02:00
TJ Rana
a61a0255a8
Modify pull request template
2021-06-12 19:34:46 -04:00
Michael Eischer
6add186867
key: Check that a new key file actually works
2021-06-12 23:09:19 +02:00
MichaelEischer
a476752962
Merge pull request #3421 from greatroar/s3-fileinfo
...
Return s3.fileInfos by pointer
2021-06-12 18:55:18 +02:00
MichaelEischer
e8d20ea32c
Merge pull request #3409 from greatroar/lchown-mknod
...
Make restic.{lchown,mknod} regular functions
2021-06-12 18:22:38 +02:00
Michael Eischer
fe43f53528
mount: enable fuse readahead
...
Apparently readahead was disabled by default. Enable readahead with the
Linux default size of 128kB. Larger values seem to have no effect.
This can speed up reading from the fuse mount by at least factor 5.
Speedup for a 1G random file stored in a local repository:
(Only one result shown, but times were quite stable, restarted restic
after each command)
$ dd if=/dev/urandom bs=1M count=1024 of=rand
$ shasum -a 256 tmp/rand
75dd9b374e712577d64672a05b8ceee40dfc45dce6321082d2c2fd51d60c6c2d tmp/rand
before: $ time shasum -a 256 fuse/snapshots/latest/tmp/rand
75dd9b374e712577d64672a05b8ceee40dfc45dce6321082d2c2fd51d60c6c2d fuse/snapshots/latest/tmp/rand
real 0m18.294s
user 0m4.522s
sys 0m3.305s
before: $ time cat fuse/snapshots/latest/tmp/rand > /dev/null
real 0m14.924s
user 0m0.000s
sys 0m4.625s
after: $ time shasum -a 256 fuse/snapshots/latest/tmp/rand
75dd9b374e712577d64672a05b8ceee40dfc45dce6321082d2c2fd51d60c6c2d fuse/snapshots/latest/tmp/rand
real 0m6.106s
user 0m3.115s
sys 0m0.182s
after: $ time cat fuse/snapshots/latest/tmp/rand > /dev/null
real 0m3.096s
user 0m0.017s
sys 0m0.241s
2021-06-12 17:07:30 +02:00
greatroar
0d4f16b6ba
Return s3.fileInfos by pointer
...
Since the fileInfos are returned in a []interface, they're already
allocated on the heap. Making them pointers explicitly means the
compiler doesn't need to generate fileInfo and *fileInfo versions of the
methods on this type. The binary becomes about 7KiB smaller on
Linux/amd64.
2021-06-07 19:48:43 +02:00
greatroar
0666c4d244
Make restic.{lchown,mknod} regular functions
...
This way, they can be inlined and dead code can be removed on Windows.
Also fixed some comments.
2021-05-27 22:51:40 +02:00
rawtaz
fdbd65485e
Merge pull request #3402 from MichaelEischer/misc-fixes
...
Various small code cleanups
2021-05-24 11:30:31 +02:00
rawtaz
2daf033156
Merge pull request #3403 from MichaelEischer/fast-cat
...
cat: only load index if really necessary
2021-05-24 11:02:10 +02:00
MichaelEischer
5dad45f005
Merge pull request #3312 from soraxas/master
...
Bump cobra and add completions for fish
2021-05-23 13:58:27 +02:00
MichaelEischer
7eb6372123
Merge pull request #3257 from greatroar/ls-json-empty
...
ls: print "size":0 for empty files in JSON
2021-05-23 13:49:56 +02:00
Michael Eischer
61b368ddea
cat: only load index if really necessary
2021-05-23 13:11:55 +02:00
Michael Eischer
fd8bce8184
backup: cleanly shutdown goroutines on error
2021-05-23 13:02:44 +02:00
greatroar
d7322a5f36
restic ls --json: print "size":0 for empty files
2021-05-21 21:06:00 +02:00
Tin Lai
9cc1ecdd45
bump cobra and add completions for fish
...
Signed-off-by: Tin Lai <oscar@tinyiu.com>
2021-05-21 13:47:52 +10:00
Alexander Neumann
af3de702c7
Merge pull request #3332 from restic/debug-1999
...
Merge `debug examine` to salvage damaged pack files
2021-05-18 09:38:40 +02:00
Alexander Neumann
226cd8d4d1
Merge pull request #3310 from MichaelEischer/copy-unstable-tree
...
`copy` raw tree blobs
2021-05-18 09:36:51 +02:00
Alexander Neumann
4cabad8c34
Merge pull request #3325 from MichaelEischer/fix-mintty-output
...
Fix windows terminal output for mintty
2021-05-18 09:29:24 +02:00
Michael Eischer
cf92c58460
Properly wrap errors in readerat helper
2021-05-17 21:08:23 +02:00
Michael Eischer
5767c65c62
delete: properly close fileChan if context is canceled
2021-05-17 21:05:54 +02:00
Michael Eischer
75c990504d
azure/gs: Fix default value in connections help text
2021-05-17 20:56:51 +02:00
Michael Eischer
5a87a0ba0a
find: use Str() to format short ids
2021-05-17 20:56:49 +02:00
Alexander Neumann
38ccddc84f
Merge pull request #3399 from oli-h/oli-h-patch-060_forget.rst
...
Sudden find: fix within first "forget" example
2021-05-17 11:39:36 +02:00
Alexander Neumann
d7b5061aa5
Merge pull request #3394 from MichaelEischer/apple-m1-build
...
Build darwin/arm64 binaries for Apple M1
2021-05-17 09:56:44 +02:00
rawtaz
27141ae87f
Merge pull request #2999 from restic/issue-2109
...
backup: Improve wording for --one-file-system description
2021-05-15 00:23:36 +02:00
Leo R. Lundgren
90d75651e6
backup: Improve wording for --one-file-system description
2021-05-15 00:06:27 +02:00
rawtaz
2a915069a8
Merge pull request #3393 from MichaelEischer/fix-filter-crash
...
filter: Fix crash for '**' pattern
2021-05-14 23:58:44 +02:00
Michael Eischer
55bea6e7a6
filter: Fix crash for '**' pattern
2021-05-14 23:50:31 +02:00
Michael Eischer
af6f6fba15
Build darwin/arm64 binaries for Apple M1
2021-05-14 23:04:45 +02:00
MichaelEischer
6d8ceefd67
Merge pull request #3373 from greatroar/simplify-limiter
...
Simplify internal/limiter
2021-05-14 20:27:31 +02:00
MichaelEischer
7349c6d338
Merge pull request #3167 from renard/limit-snapshots-list
...
Add option to limit snapshots list
2021-05-13 20:26:33 +02:00
Sébastien Gross
2a92b68e65
cmd/snapshots: Add option to limit snapshots list
...
This patch adds a `--latest` option to limit snapshots list to the n
last snapshots. It is very similar to the `--last` one but does not
limit to one entry. It also deprecates the `--last` flag usage in
favor of `--latest 1`
Output example:
$ restic snapshots --latest 2
repository 0d3eb989 opened successfully, password is correct
ID Time Host Tags Paths
------------------------------------------------------------
5a33bdcc 2020-12-14 12:30:00 local /home
73887d8e 2020-12-15 12:30:00 local /home
------------------------------------------------------------
2 snapshots
Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
2021-05-13 20:18:23 +02:00
MichaelEischer
64b00d28b1
Merge pull request #3345 from greatroar/sftp-enospc
...
Check for ENOSPC and remove broken files in SFTP
2021-05-13 20:09:38 +02:00
oli-h
23f9cb838d
Sudden find: fix within first "forget" example
...
I think this was a typo or copy/paste problem when created the doc.
Feel free to merge. Cheers
2021-05-10 08:55:42 +02:00
rawtaz
01261770bb
Merge pull request #3386 from adrian5/doc-change
...
doc: Add direct link to GitHub issue
2021-05-09 00:12:21 +02:00
adrian5
a0f1c74000
doc: Add direct link to GitHub issue
2021-05-08 21:13:11 +00:00
Alexander Neumann
be6fc02c04
Merge pull request #3376 from restic/rawtaz-doc-exclude
...
doc: Polish exclude file documentation
2021-04-27 19:25:00 +02:00
Alexander Neumann
3ce5544796
Merge pull request #3321 from restic/doc-files-from
...
doc: Improve docs for --files-from et al
2021-04-27 19:15:31 +02:00
rawtaz
556caa326f
doc: Polish exclude file documentation
2021-04-25 20:36:11 +02:00
greatroar
ae170e2b38
Simplify internal/limiter
2021-04-24 11:54:43 +02:00
rawtaz
f7316cea07
Merge pull request #3371 from cyounkins/max_unused_default
...
doc: default for --max-unused
2021-04-23 03:18:05 +02:00
Craig Younkins
32a84ab3e4
doc: default for --max-unused
2021-04-22 21:12:37 -04:00
rawtaz
6c7eabf08c
Merge pull request #3363 from davegallant/design-doc-grammar-fix
...
doc: Fix grammar in design.rst
2021-04-15 11:01:38 +02:00
Dave Gallant
4aaf10c356
doc: Fix grammar in design.rst
2021-04-14 17:25:53 -04:00
Michael Eischer
f65e1211d9
Add changelog entry
2021-04-11 20:02:09 +02:00
Michael Eischer
7cb8ea69ba
Add test to mintty pipe detection
2021-04-11 20:02:09 +02:00
Michael Eischer
80564a9bc9
Properly detect mintty output redirection
...
mintty on windows always uses pipes to connect stdout between processes
and for the terminal output. The previous implementation always assumed
that stdout connected to a pipe means that stdout is displayed on a
mintty terminal. However, this detection breaks when using pipes to
connect processes and for powershell which uses pipes when redirecting
to a file.
Now the pipe filename is queried and matched against the pattern used by
msys / cygwin when connected to the terminal. In all other cases assume
that a pipe is just a regular pipe.
2021-04-11 20:02:09 +02:00
Michael Eischer
5e6af77b7a
Unify interactive terminal detection code
...
Previously the progress bar / status update interval used
stdoutIsTerminal to determine whether it is possible to update the
progress bar or not. However, its implementation differed from the
detection within the backup command which included additional checks to
detect the presence of mintty on Windows. mintty behaves like a terminal
but uses pipes for communication.
This adds stdoutCanUpdateStatus() which calls the same terminal detection
code used by backup. This ensures that all commands consistently switch
between interactive and non-interactive terminal mode.
stdoutIsTerminal() now also returns true whenever stdoutCanUpdateStatus()
does so. This is required to properly handle the special case of mintty.
2021-04-11 20:02:09 +02:00
MichaelEischer
cc254dfefe
Merge pull request #3362 from greatroar/darwin-preallocate
...
Use FcntlFstore to preallocate on Mac
2021-04-10 22:47:56 +02:00
greatroar
23531be272
Use FcntlFstore to preallocate on Mac
2021-04-10 16:54:07 +02:00
Alexander Neumann
b922fc851b
Merge pull request #3356 from restic/rawtaz-doc-dollar-sign
...
doc: Clarify dollar sign expansion in exclude files
2021-04-09 12:30:54 +02:00
rawtaz
ccfd5f1d4a
Merge pull request #3298 from jniggemann/patch-1
...
doc: Add note about bash completion path
2021-04-09 00:17:17 +02:00
Jan Niggemann
9a1f685179
doc: Add note about bash completion path
2021-04-08 23:53:38 +02:00
rawtaz
b5e40b370c
doc: Clarify dollar sign expansion in exclude files
2021-04-08 23:46:53 +02:00
rawtaz
74c0607c92
Merge pull request #3319 from MichaelEischer/skip-prealloc-test
...
restorer: Skip preallocate test if not supported by the filesystem
2021-04-07 18:59:06 +02:00
Leo R. Lundgren
5861bb031c
doc: Improve docs for --files-from et al
2021-04-07 18:31:46 +02:00
rawtaz
c2569ff923
Merge pull request #3354 from greatroar/filechange-docs
...
docs: on Windows, the filesize must match for "unchanged"
2021-04-07 15:29:46 +02:00
greatroar
ecbe7f3a99
Backup docs: on Windows, the filesize must match for "unchanged"
2021-04-04 17:01:48 +02:00
Michael Eischer
88731d8c28
Add changelog
2021-03-27 17:08:35 +01:00
greatroar
dc88ca79b6
Handle lack of space and remove broken files in SFTP backend
2021-03-27 15:02:14 +01:00
MichaelEischer
efb10b3c40
Merge pull request #3347 from jazcap53/fix_ambiguous_warning
...
Change ambiguous Warning message
2021-03-27 12:54:07 +01:00
MichaelEischer
d456437ad1
Merge pull request #3343 from hluup/3334
...
Display 'created new cache in ' message only if output is terminal
2021-03-27 12:43:37 +01:00
MichaelEischer
4c61825249
Merge pull request #3335 from cyounkins/patch-1
...
prune --max-unused does not limit metadata repacking
2021-03-27 12:40:35 +01:00
Andrew Jarcho
9f44129c2f
Change ambiguous Warning message
...
Fixes #3346
On branch fix_ambiguous_warning
Changes to be committed:
modified: cmd_backup.go
2021-03-24 10:44:47 -04:00
Hendrik Luup
5592c17e4a
Display 'created new cache in ' message only if output is terminal
2021-03-22 11:31:08 +02:00
Craig Younkins
aa0a7b78a8
doc: prune --max-unused unlimited will still repack metadata
...
Adding minor clarification to documentation of `prune --max-unused unlimited` to indicate metadata will still be repacked. The referenced PR indicates `max-repack-size` CAN limit metadata repacking.
Ref: https://forum.restic.net/t/max-unused-unlimited-still-repacks/3661
Ref: https://github.com/restic/restic/pull/3228
2021-03-21 13:19:09 -04:00
Alexander Neumann
88a23521dd
Merge pull request #3327 from MichaelEischer/fix-s3-sanity-check
...
s3: Fix sanity check
2021-03-11 13:13:46 +01:00
MichaelEischer
e678acafcf
Merge pull request #3331 from greatroar/upgrade-sftp
...
Upgrade pkg/sftp to 1.13
2021-03-10 22:27:32 +01:00
Michael Eischer
54d58edacc
debug: fix usage for examine command
2021-03-10 22:22:33 +01:00
Michael Eischer
5975ed61f3
debug: fix linter warning
2021-03-10 21:22:53 +01:00
Michael Eischer
dc62ec5933
debug: use Printf/Warnf for output
2021-03-10 21:20:21 +01:00
Michael Eischer
547d9b384d
debug: cleanup repair code
2021-03-10 21:15:38 +01:00
greatroar
187a77fb27
Upgrade pkg/sftp to 1.13 and simplify SFTP.IsNotExist
2021-03-10 21:05:24 +01:00
Michael Eischer
fa7b9d5dfe
debug: Cleanup pack size checks
2021-03-10 20:57:14 +01:00
Michael Eischer
6774fc6454
debug: check arguments and cleanup error handling
2021-03-10 20:43:22 +01:00
Michael Eischer
096f15db5c
debug: extract examinePack function
2021-03-10 20:21:05 +01:00
Michael Eischer
84491ff40b
debug: store repaired and correct blobs
2021-03-10 20:03:44 +01:00
Alexander Neumann
b3c3121622
debug: Save raw decrypt (disregarding signature)
2021-03-10 20:03:44 +01:00
Alexander Neumann
ce4b6d0874
examine: add byte repair mode
2021-03-10 20:03:44 +01:00
Alexander Neumann
52061e817c
debug: fix percentage
2021-03-10 20:03:44 +01:00
Alexander Neumann
133ac42a0b
debug: check file content hash
2021-03-10 20:03:44 +01:00
Alexander Neumann
90f975fa1c
debug: make output less verbose
2021-03-10 20:03:44 +01:00
Alexander Neumann
086993bae1
debug: check packs not in index, implement repair
2021-03-10 20:03:44 +01:00
Alexander Neumann
d6f78163d4
Add 'debug examine' command to debug #1999
2021-03-10 20:03:44 +01:00
MichaelEischer
c9b4fadd91
Merge pull request #3294 from Achilleshiel/fix-copy-repofile
...
Add RepositoryFile2 Option for secondary repository
2021-03-08 22:50:31 +01:00
Michael Eischer
da458a55db
Cleanup comments in secondary repo test
2021-03-08 22:41:13 +01:00
Wouter Horlings
9ccdba9df6
Add repositoryFile2 option
...
The `init` and `copy` commands can now use `--repository-file2` flag and
the `$RESTIC_REPOSITORY_FILE2` environment variable.
This also fixes the conflict with the `--repository-file` and `--repo2`
flag.
Tests are added for the initSecondaryGlobalOpts function.
This adds a NOK function to the test helper functions. This NOK tests if
err is not nil, and otherwise fail the test.
With the NOK function a couple of sad paths are tested in the
initSecondaryGlobalOpts function.
In total the tests checks wether the following are passed correct:
- Password
- PasswordFile
- Repo
- RepositoryFile
The following situation must return an error to pass the test:
- no Repo or RepositoryFile defined
- Repo and RepositoryFile defined both
2021-03-08 22:41:13 +01:00
MichaelEischer
a0f9d73d44
Merge pull request #3305 from juergenhoetzel/terminal-race-condition
...
backup: In case of error also Wait() for terminal goroutine
2021-03-08 22:20:18 +01:00
Michael Eischer
427781928f
copy: test that trees with unstable json encoding are properly copied
2021-03-08 22:16:48 +01:00
Michael Eischer
2fc7abac35
copy: copy raw bytes of tree blobs
...
This avoids problems when for some reason the JSON encoding changes.
This also ensures forward compatibility with future restic versions
which might e.g. add new fields to the tree metadata.
2021-03-08 22:16:48 +01:00
Michael Eischer
f9c581f219
Tweak changelog
2021-03-08 22:11:34 +01:00
Juergen Hoetzel
18fccb5995
backup: In case of error also Wait() for terminal goroutine
...
This prevents a race condition where the final summary message can get lost.
2021-03-08 22:11:34 +01:00
Michael Eischer
2a9f0f19b6
s3: Fix sanity check
...
The sanity check shouldn't replace the error message if there is already
one.
2021-03-08 20:23:57 +01:00
rawtaz
d686fa25de
Merge pull request #3323 from restic/doc-setcap
...
doc: Clarify setcap applying only to current binary
2021-03-08 19:32:11 +01:00
rawtaz
f000f41c91
doc: Clarify setcap applying only to current binary
2021-03-08 19:21:06 +01:00
MichaelEischer
64fe733fa0
Merge pull request #3320 from MichaelEischer/fix-ci
...
Update to Go 1.16 and pin go version used for golangcil-lint
2021-03-08 19:18:10 +01:00
Michael Eischer
781378a65e
Bump go version to 1.16
2021-03-04 23:24:26 +01:00
Michael Eischer
64a9272c9a
go mod tidy for go 1.16
2021-03-04 23:18:53 +01:00
Michael Eischer
ee89e33f12
CI: Pin go version used for golint to 1.15.x
...
This prevents unexpected lint failures when a new go version is
released.
2021-03-04 23:12:48 +01:00
Michael Eischer
f0e9068ef2
go mod tidy for go 1.16
2021-03-04 22:56:08 +01:00
Michael Eischer
f70aca6f6f
restorer: Skip preallocate test if not supported by the filesystem
2021-03-04 20:33:46 +01:00
MichaelEischer
814a399e4c
Merge pull request #3283 from dennypage/master
...
Treat an empty password as a fatal error for repository init.
2021-02-28 00:49:21 +01:00
Michael Eischer
13730e3844
Add link to pr to changelog
2021-02-28 00:40:52 +01:00
Denny Page
0f41e99ea7
Treat an empty password as a fatal error for repository init.
2021-02-28 00:40:00 +01:00
MichaelEischer
6712c6de73
Merge pull request #3309 from MichaelEischer/fix-random-check-crash
...
check: Fix crash of --read-data-subset=x% on empty repository
2021-02-27 16:24:56 +01:00
Michael Eischer
9e852af5be
check: Fix crash of --read-data-subset=x% on empty repository
...
Rounding up to at least one pack file should only be done if at least a
pack file exists.
2021-02-27 16:05:36 +01:00
MichaelEischer
4baebdc6f5
Merge pull request #3308 from MichaelEischer/fix-not-a-directory
...
local: Ignore files in intermediate folders
2021-02-27 15:34:36 +01:00
Michael Eischer
a293fd9aef
local: Ignore files in intermediate folders
...
For example the data folder of a repository might contain hidden files
which caused list operations to fail.
2021-02-27 13:47:55 +01:00
TJ Rana
91e8f0e486
Clean up writing style
2021-02-16 22:19:11 -05:00
TJ Rana
d7d562b287
Clarify checklist items
2021-02-16 22:11:22 -05:00
TJ Rana
76e1ba6fd0
Clean up pull request template
2021-02-16 22:03:33 -05:00
rawtaz
8eb6a5805b
Merge pull request #3286 from MichaelEischer/fix-quiet-backup
...
backup: Correctly handle --quiet flag
2021-02-15 22:35:04 +01:00
Michael Eischer
65bd2a9a49
backup: Correctly handle --quiet flag
...
The quiet flag changes the backup output to assume a non-interactive
terminal. However, the output progress interval was not set to 0 by
default.
2021-02-15 22:14:58 +01:00
Alexander Neumann
12f0ccc237
helpers: Also push versioned image to Docker
2021-02-15 20:08:40 +01:00
Alexander Neumann
bb53fcfc0d
Set development version for 0.12.0
2021-02-14 11:44:24 +01:00
Alexander Neumann
27f241334e
Add version for 0.12.0
2021-02-14 11:44:20 +01:00
Alexander Neumann
4e99a3d650
Update manpages and auto-completion
2021-02-14 11:44:20 +01:00
Alexander Neumann
1cb1cd6f44
Generate CHANGELOG.md for 0.12.0
2021-02-14 11:44:00 +01:00
Alexander Neumann
1a34260cf0
Prepare changelog for 0.12.0
2021-02-14 11:44:00 +01:00
Alexander Neumann
13d52c88fb
Merge pull request #3282 from restic/changelog-unreleased
...
changelog: Correct and slightly polish some unreleased changelog files
2021-02-14 10:49:33 +01:00
Leo R. Lundgren
4b5ca1e914
changelog: Correct and slightly polish some unreleased changelog files
2021-02-14 00:57:54 +01:00
MichaelEischer
917f5b910a
Merge pull request #3270 from restic/fix-ls-json
...
ls: Check for non-nil error before calling panic()
2021-02-09 22:05:04 +01:00
Alexander Neumann
c0f2c1d871
ls: Check for non-nil error before calling panic()
2021-02-07 21:12:54 +01:00
Alexander Neumann
9985368d46
Merge pull request #3255 from MichaelEischer/restorer-check-error
...
restorer: Check dropped error
2021-02-03 16:10:46 +01:00
Alexander Neumann
2dd592a06c
Merge pull request #3254 from lorenz/allow-http2
...
Allow HTTP/2
2021-02-02 21:08:20 +01:00
Lorenz Brun
362338dd60
Add changelog entry
2021-02-01 20:20:34 +01:00
MichaelEischer
6ac032be64
Merge pull request #3256 from aawsome/rebuild-index-bar-done
...
rebuild-index: Add missing bar.Done()
2021-01-31 18:48:14 +01:00
MichaelEischer
0ce05d5725
Merge pull request #3236 from Wouter0100/patch-1
...
doc: Note only path-style URL support for S3
2021-01-31 18:44:58 +01:00
Michael Eischer
0aed8d47d7
docs: Fix typo in S3 URL support
2021-01-31 18:33:15 +01:00
Alexander Weiss
39a26066f7
rebuild-index: add missing bar.Done()
2021-01-31 18:28:02 +01:00
Michael Eischer
47faf69230
restorer: Check dropped error
2021-01-31 18:06:28 +01:00
MichaelEischer
b3dc127af5
Merge pull request #3207 from aawsome/filerestorer-fix-errorhandling
...
Fix error handling in filerestorer and fix #3166
2021-01-31 17:47:46 +01:00
Wouter van Os
8442c43209
doc: Update wording for S3 path-only requirement
2021-01-31 15:10:03 +01:00
Alexander Weiss
6e942693ba
Fix #3166
2021-01-31 14:22:57 +01:00
Alexander Weiss
5e22ae10f1
Add error handling for fileRestorer
2021-01-31 14:22:57 +01:00
Alexander Weiss
573221aa40
Add failing test for fileRestorer
2021-01-31 13:40:42 +01:00
MichaelEischer
b8550a21f2
Merge pull request #3253 from restic/improve-prune-error
...
prune: Improve error message for missing files
2021-01-31 12:03:20 +01:00
Alexander Neumann
027a51529d
prune: Improve error message for missing files
...
This commit changes the error message so that a list of file names is
printed. Before, just the raw map was printed, which is not a great user
interface.
2021-01-31 11:31:27 +01:00
Lorenz Brun
5427119205
Allow HTTP/2
2021-01-31 02:44:30 +01:00
Alexander Neumann
f647614e24
Merge pull request #3248 from MichaelEischer/backend-cleanups
...
Backend code and test cleanups
2021-01-30 21:37:33 +01:00
Michael Eischer
e0867c9682
backend: try to cleanup test leftovers
2021-01-30 21:23:20 +01:00
Michael Eischer
f740b2fb23
mem: check upload length before storing upload
2021-01-30 21:23:20 +01:00
Alexander Neumann
0e5f2fff71
Merge pull request #3243 from restic/fix-scanner-overlap
...
backup: Fix total size for overlapping targets
2021-01-30 21:17:21 +01:00
MichaelEischer
99228be623
Merge pull request #3250 from restic/add-golangci-lint-config
...
Add golangci lint config, add many error checks
2021-01-30 21:09:38 +01:00
Alexander Neumann
04ca69cc78
Address issues reported by golint
2021-01-30 20:45:57 +01:00
Alexander Neumann
f867e65bcd
Fix issues reported by staticcheck
2021-01-30 20:43:53 +01:00
Alexander Neumann
a00e27adf6
Add entry to changelog
2021-01-30 20:22:25 +01:00
Alexander Neumann
0858fbf6aa
Add more error handling
2021-01-30 20:19:47 +01:00
Alexander Neumann
aef3658a5f
Address review comments
2021-01-30 20:02:37 +01:00
Alexander Neumann
200f09522d
Add more error checks
2021-01-30 20:02:37 +01:00
Alexander Neumann
cbd88c457a
backup: Improve error handling
2021-01-30 20:02:37 +01:00
Alexander Neumann
1a0eb05bfa
errcheck: Add more error checks
2021-01-30 20:02:37 +01:00
Alexander Neumann
3c753c071c
errcheck: More error handling
2021-01-30 20:02:37 +01:00
Alexander Neumann
16313bfcc9
errcheck: Add error check for MergeFinalIndexes()
2021-01-30 20:02:37 +01:00
Alexander Neumann
75f53955ee
errcheck: Add error checks
...
Most added checks are straight forward.
2021-01-30 20:02:37 +01:00
Alexander Neumann
1632a84e7b
Add a section explaining golangci-lint
2021-01-30 20:02:37 +01:00
Alexander Neumann
b3d5bf7c99
Update golangci-lint version
2021-01-30 20:02:37 +01:00
Alexander Neumann
57627a307f
Add config for golangci-lint
2021-01-30 20:02:37 +01:00
Alexander Neumann
6ab7d49a03
Merge pull request #3251 from MichaelEischer/rest-dropped-error
...
rest: handle dropped error in save operation
2021-01-30 19:44:50 +01:00
Michael Eischer
a53778cd83
rest: handle dropped error in save operation
2021-01-30 19:25:04 +01:00
Alexander Neumann
dd94efb307
Merge pull request #3249 from MichaelEischer/fix-dropped-gs-error
...
gs: Don't drop error when finishing upload
2021-01-30 16:24:57 +01:00
Michael Eischer
8a486eafed
gs: Don't drop error when finishing upload
...
The error returned when finishing the upload of an object was dropped.
This could cause silent upload failures and thus data loss in certain
cases. When a MD5 hash for the uploaded blob is specified, a wrong
hash/damaged upload would return its error via the Close() whose error
was dropped.
2021-01-30 13:31:32 +01:00
MichaelEischer
4d576c2f79
Merge pull request #3217 from M4a1x/read-data-subset-fix
...
Fix missing rand seed for restic check --read-data-subset=x%
2021-01-29 23:34:13 +01:00
Max Stabel
f9e1fa26ff
Fix missing rand seed for restic check --read-data-subset=x%
2021-01-29 23:18:35 +01:00
MichaelEischer
fb3cf3f885
Merge pull request #3245 from aawsome/prune-fix-statistics-cacheable
...
prune: Fix statistics for --repack-cacheable-only
2021-01-29 23:14:49 +01:00
Alexander Weiss
e08e65dc30
prune: Simplify logic selecting packs to repack
2021-01-29 22:27:22 +01:00
Alexander Weiss
daeb4cdf8f
prune: Fix statistics for --repack-cacheable-only
2021-01-29 22:27:22 +01:00
Alexander Neumann
cdd704920d
azure: Pass data length to Azure libray
...
The azureAdapter was used directly without a pointer, but the Len()
method was only defined with a pointer receiver (which means Len() is
not present on a azureAdapter{}, only on a pointer to it).
2021-01-29 21:08:41 +01:00
Alexander Neumann
bbdf18c4a2
Merge pull request #3176 from MichaelEischer/backend-content-length
...
Pass upload size to backends and sanity check it
2021-01-29 20:33:44 +01:00
Michael Eischer
1f583b3d8e
backend: test that incomplete uploads fail
2021-01-29 13:51:53 +01:00
Michael Eischer
c73316a111
backends: add sanity check for the uploaded file size
...
Bugs in the error handling while uploading a file to the backend could
cause incomplete files, e.g. https://github.com/golang/go/issues/42400
which could affect the local backend.
Proactively add sanity checks which will treat an upload as failed if
the reported upload size does not match the actual file size.
2021-01-29 13:51:51 +01:00
Michael Eischer
4526d5d197
swift: explicitly pass upload size to library
...
This allows properly setting the content-length which could help the
server-side to detect incomplete uploads.
2021-01-29 13:50:46 +01:00
Michael Eischer
dca9b6f5db
azure: explicitly pass upload size
...
Previously the fallback from the azure library was to read the whole
blob into memory and use that to determine the upload size.
2021-01-29 13:50:46 +01:00
Alexander Neumann
a16ce65295
Merge pull request #3244 from MichaelEischer/better-damage-reports
...
Print more details about possible repository damages
2021-01-29 11:45:45 +01:00
Alexander Neumann
5c41120c70
Add entry to changelog
2021-01-29 11:31:36 +01:00
Alexander Neumann
5c617859ab
backup/scanner: Fix total size for overlapping targets
...
Before, the scanner would could files twice if they were included in the
list of backup targets twice, e.g. `restic backup foo foo/bar` would
could the file `foo/bar` twice.
This commit uses the tree structure from the archiver to run the
scanner, so both parts see the same files.
2021-01-29 11:31:36 +01:00
Alexander Neumann
81211750ba
archiver/tree: Introduce functions Leaf() and NodeNames()
2021-01-29 11:11:28 +01:00
rawtaz
de7e3a0648
Merge pull request #2823 from greatroar/trust-mtime
...
Add --ignore-ctime flag to backup and document change detection
2021-01-29 00:02:19 +01:00
greatroar
6bd8a2faaa
backup: Add --ignore-ctime option and document change detection
2021-01-28 23:42:10 +01:00
Michael Eischer
58b5679f14
prune: reword missing blobs error
...
The previous wording could be understood such that the prune run did
damage the repository.
2021-01-28 21:48:24 +01:00
Michael Eischer
7b8886c052
prune: report missing but unneeded pack files
...
This indicates a damaged repository so add some output to help with
debugging.
2021-01-28 21:46:01 +01:00
Michael Eischer
ff95999246
rebuild-index: report added/removed/reindexed files
...
This should help with investigating missing pack files.
2021-01-28 21:46:01 +01:00
Michael Eischer
b71c52797a
find: correctly expand multiple blob ids
...
For example `restic find --show-pack-id --blob f78dc991 5b9e4366 ddd8c7d4`
would previously only expand one blob if all of them belong to the same
file.
2021-01-28 21:21:54 +01:00
MichaelEischer
82140967d3
Merge pull request #3228 from aawsome/prune-all-trees
...
prune: Remove all unused trees
2021-01-28 21:04:27 +01:00
MichaelEischer
43cb26010a
Merge pull request #3242 from greatroar/fprintln
...
internal/ui/termstatus: Use Fprintln to get a newline
2021-01-28 20:34:06 +01:00
MichaelEischer
35033d9b79
Merge pull request #3177 from MichaelEischer/fix-2759
...
prune: don't print stacktrace on console
2021-01-28 20:28:45 +01:00
Alexander Neumann
84822d44d4
Merge pull request #2536 from MatthewVance/threat-model
...
Update threat model
2021-01-28 14:26:03 +01:00
Matthew Vance
58c7f4694d
Update threat model
2021-01-28 14:20:42 +01:00
Alexander Neumann
4d40c70214
Merge pull request #3211 from MichaelEischer/sftp-speedup
...
Speed-up caching/pack download via SFTP
2021-01-28 14:16:55 +01:00
Alexander Neumann
44169d0dc4
Merge pull request #3205 from MichaelEischer/fix-quiet-verbose
...
Properly check that --quiet and --verbose are not combined
2021-01-28 13:53:31 +01:00
Alexander Neumann
6aa7e9f9c6
Merge pull request #3174 from MichaelEischer/parallelize-lock-loading
...
Parallelize lock file loading
2021-01-28 13:52:12 +01:00
Alexander Neumann
bdfedf1f5b
Merge pull request #3173 from MichaelEischer/unify-index-loading
...
Unify index loading
2021-01-28 13:50:42 +01:00
greatroar
b9cfe6f68a
internal/ui/termstatus: Use Fprintln to get a newline
2021-01-28 13:30:10 +01:00
Alexander Neumann
72eec8c0c4
Merge pull request #3106 from MichaelEischer/parallel-tree-walk
...
Parallelize tree walk in prune and copy and add progress bar to check
2021-01-28 12:06:42 +01:00
Michael Eischer
68608a89ad
restic: add comment about StreamTrees shutdown
2021-01-28 11:10:50 +01:00
Michael Eischer
1e306be000
Add changelog entry
2021-01-28 11:10:50 +01:00
Michael Eischer
ddb7697d29
restic: Test progress reporting of StreamTrees
2021-01-28 11:10:50 +01:00
Michael Eischer
313ad0e32f
progress/counter: Fix test for final report call
2021-01-28 11:10:50 +01:00
Michael Eischer
e2b0072441
check: add progress bar to the tree structure check
2021-01-28 11:10:50 +01:00
Michael Eischer
505f8a2229
progress/counter: Support updating the progress bar maximum
2021-01-28 11:10:47 +01:00
Michael Eischer
eda8c67616
restic: let FindUsedBlobs handle multiple snapshots at once
2021-01-28 11:08:43 +01:00
Michael Eischer
258ce0c1e5
parallel: report progress for StreamTrees
...
This assigns an id to each tree root and then keeps track of how many
tree loads (i.e. trees referenced for the first time) are pending per
tree root. Once a tree root and its subtrees were fully processed there
are no more pending tree loads and the tree root is reported as
processed.
2021-01-28 11:08:43 +01:00
Michael Eischer
3d6a3e2555
copy: Remove treeCloner struct
2021-01-28 11:08:43 +01:00
Michael Eischer
0caad1e890
copy: parallelize tree walk
2021-01-28 11:08:43 +01:00
Michael Eischer
f2a1b125cb
restic: Actually parallelize FindUsedBlobs
2021-01-28 11:08:43 +01:00
Michael Eischer
6e03f80ca2
check: Split the parallelized tree loader into a reusable component
...
The actual code change is minimal
2021-01-28 11:08:43 +01:00
Michael Eischer
1d7bb01a6b
check: Cleanup tree loading and switch to use errgroup
...
The helper methods are now wired up in the Structure method.
2021-01-28 11:08:43 +01:00
Alexander Neumann
a4689eb3b9
Merge pull request #3199 from MichaelEischer/non-interactive-counter
...
Don't print progress on non-interactive terminals
2021-01-28 10:53:38 +01:00
Alexander Neumann
c5a66e9181
ui: Simlify channel receive
2021-01-28 10:42:02 +01:00
Wouter van Os
b5972f184c
doc: Note only path-style URL support for S3
...
This adds a node to mention that currently only path-style URL's are supported for S3, as per code at:
- aa0faa8c7d/internal/backend/s3/config.go (L42-L45)
- aa0faa8c7d/internal/backend/s3/config.go (L48-L62)
2021-01-23 16:54:08 +01:00
Alexander Weiss
d7dc19a496
prune: Always repack packs containing tree blobs
2021-01-15 16:42:04 +01:00
Michael Eischer
f3442ce8a5
Test that WriteTo of a backend's Load remains accessible
2021-01-03 22:23:53 +01:00
Michael Eischer
678e75e1c2
sftp: enforce use of optimized upload method
...
ReadFrom was already used by Save before, this just ensures that this
won't accidentally change in the future.
2021-01-03 22:23:53 +01:00
Michael Eischer
6b5b29dbee
limiter: add unit tests
2021-01-03 22:23:53 +01:00
Michael Eischer
f35f2c48cd
limiter: support WriteTo in LimitBackend for read rate limiting
2021-01-03 22:23:53 +01:00
Michael Eischer
bcb852a8d0
hashing: support WriteTo in the reader
2021-01-03 22:23:53 +01:00
MichaelEischer
aa0faa8c7d
Merge pull request #3208 from restic/add-mips
...
Add mips* architectures to CI and release
2021-01-03 16:06:10 +01:00
MichaelEischer
f7ec263a22
Merge pull request #3109 from aawsome/optimize-filerestorer
...
restore: Don't save pack content in memory
2021-01-03 14:53:41 +01:00
Alexander Neumann
7d665fa1f4
Add entry to changelog
2021-01-03 14:41:11 +01:00
Michael Eischer
69d5b4c36b
restorer: lower-case variable name
2021-01-03 13:55:59 +01:00
Alexander Neumann
36db248e30
Split cross compilation targets into two jobs
2021-01-03 10:50:54 +01:00
Alexander Neumann
eb72b10f55
Add mips* architectures to CI and release
2021-01-03 10:41:54 +01:00
Alexander Neumann
622f4c7daa
Run cloud backends also on pushes
2021-01-03 10:38:10 +01:00
Alexander Neumann
f8c50394d6
Revert "Run cloud tests if secrets are available"
...
This reverts commit aa648bdcac
.
2021-01-03 10:28:01 +01:00
Alexander Neumann
aa648bdcac
Run cloud tests if secrets are available
2021-01-03 10:26:55 +01:00
Alexander Neumann
e8abc79ce9
Add hints for keeping the list of architectures in sync
2021-01-01 10:09:04 +01:00
Alexander Weiss
34a33565c8
Fix loadBlob in filerestorer
2021-01-01 08:06:04 +01:00
Alexander Weiss
7409225fa8
Add filerestorer test where only parts of pack are used
2021-01-01 07:24:46 +01:00
Alexander Weiss
07b3f65a6f
filesrestorer: Re-use buffer
2021-01-01 07:24:46 +01:00
Alexander Weiss
3e0acf1395
restore: Don't save (part of) pack in memory
2021-01-01 07:24:46 +01:00
Michael Eischer
97388b3504
Properly check that --quiet and --verbose are not combined
...
If --verbose is specified once, then globalOptions.Verbose == 1.
Previously --quiet --verbose would silently ignore the --verbose flag.
2020-12-30 21:24:18 +01:00
Alexander Neumann
8b84c96d9d
Merge pull request #3204 from MichaelEischer/archiver-tomb-race
...
archiver: fix race condition during worker startup
2020-12-30 20:04:20 +01:00
Michael Eischer
debc4a3a99
archiver: fix race condition during worker startup
...
When the tomb is created with a canceled context, then the workers
started via `t.Go` exist nearly immediately. Once for the first time all
started goroutines have been stopped, it is not allowed to issue further
calls to `t.Go`. This is a problem when the started goroutines exit
immediately, as for example the first goroutine might already have
stopped before starting the second one, which is not allowed as once the
first goroutines has stopped no goroutines were running.
To fix this race condition the startup and main task of the archiver now
also run within a `t.Go` function. This also allows unifying the error
handling as it is no longer necessary to distinguish between errors
returned by the workers or the saveTree processing. The tomb now just
returns the first error encountered, which should also be the most
descriptive one.
2020-12-30 17:31:22 +01:00
MichaelEischer
e1efc193e1
Merge pull request #3139 from aawsome/prune-healing
...
prune: Add healing of repository in some situations
2020-12-29 22:17:27 +01:00
Alexander Weiss
f0113139ea
prune: Correct error message
2020-12-29 20:20:05 +01:00
Alexander Weiss
f6df94a50e
prune: Add self-healing
...
Allow prune to heal situations where blobs in the index are missing or
the corresponding packfiles are damaged if those blobs are not needed.
2020-12-29 20:20:05 +01:00
MichaelEischer
31e56f1ad5
Merge pull request #3197 from SkYNewZ/fix/3183
...
Fix tag handling for multiple tag lists
2020-12-29 18:44:38 +01:00
MichaelEischer
7fda2f2ad8
Merge pull request #3134 from greatroar/unlock-warn
...
Warn when unlock fails instead of returning an error
2020-12-29 18:30:08 +01:00
greatroar
dec5008369
Warn when unlock fails instead of returning an error
...
Only one caller was checking the error.
2020-12-29 17:48:20 +01:00
Quentin Lemaire
873505ed3b
Update related changelog
2020-12-29 17:12:46 +01:00
Quentin Lemaire
25ecf9eafb
fix(cmd_tag): Use restic.TagLists
2020-12-29 17:12:46 +01:00
Quentin Lemaire
e88f3fb80c
fix(cmd_backup): Use restic.TagLists
2020-12-29 17:12:46 +01:00
Alexander Neumann
b2efa0af39
Merge pull request #3164 from MichaelEischer/improve-context-cancel
...
Improve context cancel handling in archiver and backends
2020-12-29 17:03:42 +01:00
Michael Eischer
25f4acdaa8
Add changelog entry
2020-12-29 16:32:18 +01:00
Michael Eischer
cff4955a48
ui: remove dead struct member
2020-12-29 16:32:18 +01:00
Michael Eischer
05a987b07c
Support values less than 1 for RESTIC_PROGRESS_FPS
...
For example set the variable to 0.016666 to print the progress once per
minute.
2020-12-29 16:32:18 +01:00
Michael Eischer
92da5168e1
ui: force backup progress update on signal
2020-12-29 16:32:18 +01:00
Michael Eischer
34afc93ddc
ui/progress: extract signal handling into own package
2020-12-29 16:32:18 +01:00
Michael Eischer
023eea6463
ui: don't shorten non-interactive progress output
2020-12-29 16:32:18 +01:00
Michael Eischer
684600cf42
ui: update status for the backup command on non-interactive terminals
...
Allow the backup command to print status on non-interactive terminals.
The output is disabled by setting a MinUpdatePause == 0.
2020-12-29 16:03:43 +01:00
Michael Eischer
85fe5feadb
Unify progress report frequency calculation
2020-12-29 16:03:43 +01:00
Michael Eischer
969141b5e9
Honor RESTIC_PROGRESS_FPS env variable on non-interactive terminals
...
This makes it possible to use the environment variable to also get
regular progress updates on non-interactive terminals.
2020-12-29 16:03:43 +01:00
Michael Eischer
13ce981794
ui: cleanup backup status shutdown
2020-12-29 16:03:43 +01:00
Michael Eischer
c2ef049f1b
ui/progress: don't print progress on non-interactive terminals
...
This reverts to the old behavior of not printing progress updates on
non-interactive terminals. It was accidentally changed in #3058 .
2020-12-29 16:03:43 +01:00
MichaelEischer
a488d4c847
Merge pull request #2833 from greatroar/aix
...
AIX port
2020-12-29 12:32:50 +01:00
Alexander Neumann
4133b1ea65
Synchronize OS and architectures for testing
2020-12-29 11:11:50 +01:00
Alexander Neumann
46d2ca5095
Remove old test script
2020-12-29 11:02:48 +01:00
Quentin Lemaire
334d8ce724
feat(tags): Create Flatten() method
2020-12-29 10:59:46 +01:00
Alexander Neumann
c661518df9
Merge pull request #2793 from greatroar/rclone-doc
...
Clarify rclone-over-SSH docs
2020-12-29 10:46:34 +01:00
Michael Eischer
0d81f16343
Add AIX as cross-compile target to CI
2020-12-29 01:35:01 +01:00
greatroar
3b09ae9074
AIX port
2020-12-29 01:35:01 +01:00
greatroar
18531e3d6f
Portability fixes to internal/restic
...
syscall.Mknod is not available on AIX.
2020-12-29 01:35:01 +01:00
Michael Eischer
ca07317815
add changelog
2020-12-28 21:06:47 +01:00
Michael Eischer
d0ca8fb0b8
backend: test that a canceled context prevents RetryBackend operations
2020-12-28 21:06:47 +01:00
Michael Eischer
08b7f2b58d
archiver: test that context canceled error is not dropped
2020-12-28 21:06:47 +01:00
Michael Eischer
e483b63c40
retrybackend: Fail operations when context is already canceled
...
Depending on the used backend, operations started with a canceled
context may fail or not. For example the local backend still works in
large parts when called with a canceled context. Backends transfering
data via http don't work. It is also not possible to retry failed
operations in that state as the RetryBackend will abort with a 'context
canceled' error.
Ensure uniform behavior of all backends by checking for a canceled
context by checking for a canceled context as a first step in the
RetryBackend. This ensures uniform behavior across all backends, as
backends are always wrapped in a RetryBackend.
2020-12-28 21:06:47 +01:00
Michael Eischer
fc60b560ba
archiver: Let saveTree report a canceled context as an error
...
If the context was canceled then saveTree might receive a treeID or not
depending on the timing. This could cause saveTree to incorrectly return
a nil treeID as valid. Fix this always returning an error when the
context was canceled in the meantime.
2020-12-28 21:06:47 +01:00
Michael Eischer
736e964317
archiver: Don't loose error if background context is canceled
...
A canceled background context lets the blob/tree/fileSavers exit
without reporting an error. The error handling previously replaced
a 'context canceled' error received by the main backup method with
the error reported by the savers. However, in case of a canceled
background context that error is nil, causing restic to loose the
error and save a snapshot with a nil tree.
2020-12-28 21:06:47 +01:00
MichaelEischer
9c41e4a343
Merge pull request #3192 from DRON-666/fix-options
...
Fix formatting of `restic options` command
2020-12-23 22:53:40 +01:00
DRON-666
332b1896d1
Some `options` fixes
...
Add tests for bool type.
Fix subtle bug in TestOptionsApplyInvalid.
Fix options list formatting.
2020-12-23 23:26:04 +03:00
MichaelEischer
cb6b0f6255
Merge pull request #3181 from tWido/feature-no-retry-permission
...
Don't retry when "Permission denied" occurs in local backend
2020-12-23 20:24:29 +01:00
MichaelEischer
1e73aac610
Merge pull request #3179 from aawsome/check-filesize
...
check: Remove filesize counter
2020-12-23 20:22:02 +01:00
Alexander Weiss
2a1add7538
check: remove file size counter
2020-12-23 02:34:31 +01:00
tWido
7dab113035
Don't retry when "Permission denied" occurs in local backend
2020-12-22 23:41:12 +01:00
MichaelEischer
8efb874f48
Merge pull request #3148 from aawsome/simplify-index-rebuild
...
rebuild-index: code simplification
2020-12-22 23:27:53 +01:00
Michael Eischer
de99207046
repository: tweak comment for packs method
2020-12-22 23:01:58 +01:00
Alexander Weiss
d8d2cc6dd9
Simplify rebuild-index code
2020-12-22 23:01:58 +01:00
Alexander Weiss
68b74e359e
Count packs directly in RebuildIndexFiles
2020-12-22 23:01:58 +01:00
Michael Eischer
b9f5d3fe13
repository: Add test for ForAllIndexes
2020-12-22 22:36:18 +01:00
Michael Eischer
a12c5f1d37
repository: move otherwise unused LoadIndex to tests
2020-12-22 22:36:18 +01:00
Michael Eischer
24474a36f4
repository: deduplicate index loading implementation
2020-12-22 22:36:18 +01:00
Michael Eischer
ccc84af73d
debug/list: parallelize index loading
2020-12-22 22:36:18 +01:00
Michael Eischer
96904f8972
check: extract parallel index loading
2020-12-22 22:36:18 +01:00
MichaelEischer
69f9d269eb
Merge pull request #3189 from restic/fix-counter-segfault
...
ui/progress: Use mutex instead of atomic
2020-12-22 22:14:08 +01:00
rawtaz
ec59c73489
Merge pull request #3188 from restic/forget-prune-dry-run
...
forget: Enable --dry-run together with --prune
2020-12-22 21:09:15 +01:00
Alexander Neumann
6c514adb8a
ui/progress: Use mutex instead of atomic
...
The counter value needs to be aligned to 64 bit in memory for the
atomic functions to work on some platform (such as 32 bit ARM).
The atomic package says in its documentation:
> These functions require great care to be used correctly. Except for
> special, low-level applications, synchronization is better done with
> channels or the facilities of the sync package.
This commit replaces the atomic functions with a simple sync.Mutex, so
we don't have to care about alignment.
2020-12-22 21:03:27 +01:00
Alexander Neumann
edf89e1c74
forget: Enable --dry-run together with --prune
2020-12-22 20:58:02 +01:00
Alexander Neumann
f7c7c2f730
Merge pull request #3175 from MichaelEischer/fix-3099-changelog
...
Fix changelog for #3099
2020-12-20 11:29:57 +01:00
Michael Eischer
cfea79d0c5
prune: don't print stacktrace on console
2020-12-19 14:28:48 +01:00
Michael Eischer
5cd40f8b58
Add changelog
2020-12-19 11:38:38 +01:00
MichaelEischer
d32949ee54
Merge pull request #3081 from DRON-666/dump-zip
...
Add zip support to the `dump` command
2020-12-19 11:33:33 +01:00
DRON-666
83b10dbb12
Deduplicate dumper closing logic
2020-12-19 02:42:46 +03:00
DRON-666
e136dd8696
Deduplicate test code
2020-12-19 02:06:54 +03:00
DRON-666
33adb58817
Minor fixes and linter suggestions
2020-12-19 02:04:17 +03:00
DRON-666
da9053b184
Some gramma fixes in documentation
2020-12-19 01:16:15 +03:00
DRON-666
ef1aeb8724
dump: Update docs and changelog
2020-12-19 01:09:47 +03:00
DRON-666
2ca76afc2b
dump: Add new option --archive
2020-12-19 01:09:47 +03:00
DRON-666
89ab6d557e
dump: Add zip dumper
2020-12-19 01:09:47 +03:00
DRON-666
0256f95994
dump: Split tar and walk logic
2020-12-19 01:09:47 +03:00
Michael Eischer
bfadc82a20
Fix changelog for #3099
2020-12-18 20:58:24 +01:00
Michael Eischer
34b6130a0e
restic: parallelize lock file loading
2020-12-18 20:46:16 +01:00
MichaelEischer
22260d130d
Merge pull request #3170 from greatroar/dont-retry
...
Don't retry permanent errors in backends
2020-12-17 23:29:56 +01:00
greatroar
9341a83b05
Changelog entry for #2453 and #3170
2020-12-17 23:15:48 +01:00
greatroar
66d904c905
Make invalid handles permanent errors
2020-12-17 12:47:53 +01:00
greatroar
746dbda413
Mark "ssh exited" errors in SFTP as permanent
2020-12-17 12:43:09 +01:00
greatroar
f7784bddb3
Don't retry when "no space left on device" in local backend
...
Also adds relevant documentation to the restic.Backend interface.
2020-12-17 12:43:09 +01:00
Alexander Neumann
1cdd38d9e0
Merge pull request #3165 from restic/2747-doc-forget
...
doc: Clarify calendar boundaries for --keep-* options
2020-12-16 08:38:03 +01:00
Leo R. Lundgren
b3c0d2f45b
doc: Clarify calendar boundaries for --keep-* options
2020-12-15 12:43:06 +01:00
Peter Albrecht
cd25e36811
Add PGP fingerprint to 020_installation.rst
...
I like the idea of verifying the integrity of applications, I download from the internet. So I was very happy to see that restic does provide SHA256-checksums which are signed with the maintainers PGP key.
The only thing I miss: I could not find a direct way to download the used PGP key and verify the keys fingerprint.
Doing some searches, I found:
* https://github.com/restic/rest-server/issues/121
* https://restic.net/blog/2015-09-16/verifying-code-archive-integrity/
To help other restic users, I think you should add information about your PGP key/fingerprint to this installation doc, too. To save you some precious time, I created a draft, how this doc might be expanded, in this pull-request. You are free to accept it or change the text to your liking.
I copied the key/fingerprint text from: ``restic/restic/master/doc/090_participating.rst``
Thank you for your work in restic!
2020-12-13 17:00:00 +01:00
Alexander Neumann
e96677cafb
Merge pull request #3158 from MichaelEischer/support-swift-auth-id-variables
...
swift: Add support for id based keystone v3 auth parameters
2020-12-12 16:27:38 +01:00
Michael Eischer
1d69341e88
swift: Add support for id based keystone v3 auth parameters
...
This adds support for the following environment variables, which were
previously missing:
OS_USER_ID User ID for keystone v3 authentication
OS_USER_DOMAIN_ID User domain ID for keystone v3 authentication
OS_PROJECT_DOMAIN_ID Project domain ID for keystone v3 authentication
OS_TRUST_ID Trust ID for keystone v3 authentication
2020-12-11 19:22:34 +01:00
Alexander Neumann
36c5d39c2c
Fix issues reported by semgrep
2020-12-11 09:41:59 +01:00
Alexander Neumann
7facc8ccc1
Merge pull request #2505 from aawsome/fix-repo-configfile
...
Fix repo configfile
2020-12-07 07:52:37 +01:00
Alexander Neumann
ba31c6fdaa
Merge pull request #3150 from MichaelEischer/fix-windows-redir-output
...
termstatus: Fix canUpdateStatus detection for redirected output on windows
2020-12-06 21:17:27 +01:00
Alexander Neumann
b58799d83a
Merge pull request #3152 from MichaelEischer/fix-backup-background-hang
...
backup: Fix shutdown hang when running in the background on linux
2020-12-06 21:16:00 +01:00
Alexander Neumann
0d5b764f90
Merge pull request #3130 from aawsome/snapshots-parallel
...
Make loading snapshots parallel
2020-12-06 21:08:18 +01:00
Alexander Weiss
d6b3859e48
Add changelog
2020-12-06 19:29:18 +01:00
Michael Eischer
b48f579530
termstatus: Fix canUpdateStatus detection for redirected output
...
The canUpdateStatus check was simplified in #2608 , but it accidentally flipped
the condition. The correct check is as follows: If the output is a pipe then
restic probably runs in mintty/cygwin. In that case it's possible to
update the output status. In all other cases it isn't.
This commit inverts to condition again to offer the previous and correct
behavior.
2020-12-06 19:02:42 +01:00
Michael Eischer
401ef92c5f
backup: Fix shutdown hang when running in the background
...
On shutdown the backup commands waits for the terminal output goroutine
to stop. However while running in the background the goroutine ignored
the canceled context.
2020-12-06 18:53:41 +01:00
Alexander Weiss
e329623771
Remove LoadAllSnapshots
2020-12-06 05:22:27 +01:00
Alexander Weiss
26f85779be
Parallelize ForAllSnapshots
2020-12-06 05:09:58 +01:00
Alexander Weiss
5b9ee56335
Add ForAllSnapshots
2020-12-06 05:04:21 +01:00
MichaelEischer
3264eae9f6
Merge pull request #3149 from aawsome/fix-rebuild-index
...
Bugfix for rebuild-index
2020-12-05 22:18:22 +01:00
Alexander Weiss
83c8a9b058
Bugfix: packSizeFromList should save size from List()
2020-12-05 20:58:36 +01:00
Alexander Neumann
43cf301450
Merge pull request #3141 from MichaelEischer/fix-fuse-dir-owner
...
fuse: Properly set uid/gid for directories
2020-12-01 16:06:24 +01:00
Michael Eischer
d05c88a5d6
fuse: Properly set uid/gid for directories
...
In #2584 this was changed to use the uid/gid of the root node. This
would be okay for the top-level directory of a snapshot, however, this
change also applied to normal directories within a snapshot. This
change reverts the problematic part and adds a test that directory
attributes are represented correctly.
2020-11-30 23:41:49 +01:00
Alexander Neumann
058b102db0
Merge pull request #3138 from MichaelEischer/fix-debug-build
2020-11-30 12:27:51 +01:00
rawtaz
fcebc7d250
Merge pull request #3135 from restic/manual
...
doc: Fix misc missing/incorrect text in manual
2020-11-29 19:20:51 +01:00
MichaelEischer
f2959127b6
Merge pull request #3065 from greatroar/local-subdirs
...
Don't recurse in local backend's List if not required
2020-11-29 19:03:59 +01:00
Leo R. Lundgren
61460dee52
doc: Fix misc missing/incorrect text in manual
2020-11-29 18:59:24 +01:00
Michael Eischer
54a6d98945
Enable debug builds for CI
2020-11-29 18:47:00 +01:00
Michael Eischer
f72f6c9c80
Fix debug build
2020-11-29 18:44:36 +01:00
MichaelEischer
52b98f7f95
Merge pull request #3017 from greatroar/files-from0
...
Add backup options --files-from-verbatim and --files-from-raw
2020-11-29 18:15:21 +01:00
Alexander Neumann
04d856e601
Merge pull request #3136 from restic/rawtaz-copy-doc
...
doc: Emphasize double transfer and duplication in copy command
2020-11-29 13:57:10 +01:00
Alexander Neumann
a7b49c4889
Merge pull request #3119 from restic/keep-mountpoints
...
Keep mountpoints as empty directories for --one-file-system
2020-11-29 11:22:12 +01:00
rawtaz
a568211b98
Merge pull request #3128 from vrenaville/tzdata
...
[FIX] Timezone in docker image
2020-11-28 22:43:18 +01:00
Leo R. Lundgren
f70b10d0ee
doc: Emphasize double transfer and duplication in copy command
2020-11-28 19:49:34 +01:00
greatroar
55bf76ba0c
backup: Add --files-from-{verbatim,raw} options
2020-11-28 18:22:31 +01:00
Alexander Neumann
162117c42c
Add changelog
2020-11-28 17:00:31 +01:00
Alexander Neumann
82ae942965
backup: Keep mountpoints for --one-file-system
...
When a file system is mounted at a directory, lstat() returns attributes
of the root node of the mounted file system, including the device ID of
the other file system. The previous code used when --one-file-system is
specified excluded the directory itself because of that.
This commit changes the code so that mountpoints are kept as empty
directories, its attributes set to the root note of the mounted file
system. The behavior mimics `tar`, which does the same.
2020-11-28 17:00:31 +01:00
Alexander Neumann
f576d3d826
Add tests
2020-11-28 17:00:31 +01:00
Alexander Neumann
037f0a4c91
Refactor device ID checking
2020-11-28 17:00:31 +01:00
Alexander Neumann
2f9346a5af
Merge pull request #3125 from metalsp0rk/cat-respect-no-lock
...
Make restic cat respect --no-lock
2020-11-28 12:45:59 +01:00
Alexander Neumann
a3105799c9
Print warning when unlocking the repo fails
2020-11-28 11:31:25 +01:00
Kyle Brennan
666768cd17
cat: Respect --no-lock flag
2020-11-28 11:30:02 +01:00
vrenaville
adc7a6555f
[FIX] Timezone in docker image
2020-11-27 08:05:19 +01:00
Alexander Neumann
9a97095a4c
Merge pull request #3120 from aawsome/blob-implementation
...
Blob implementation
2020-11-22 20:59:30 +01:00
Alexander Weiss
aa7a5f19c2
Use BlobHandle in index methods
2020-11-22 20:41:12 +01:00
Alexander Weiss
e3013271a6
Harmonize naming
2020-11-22 20:41:12 +01:00
Alexander Weiss
92bd448691
Make BlobHandle substruct of Blob
2020-11-22 20:41:10 +01:00
Alexander Neumann
c844580e0f
Merge pull request #3101 from aawsome/packsizes
...
Compute packsizes in MasterIndex
2020-11-22 15:49:19 +01:00
Alexander Weiss
67c938f232
Use PackSize in prune
2020-11-21 22:13:54 +01:00
Alexander Weiss
a851c53cbe
Use PackSize in checker
2020-11-21 22:13:54 +01:00
Alexander Weiss
4960b841e6
Use PackSize in rebuild-index
2020-11-21 22:13:54 +01:00
Alexander Weiss
ce5d630681
Add MasterIndex.PackSize()
2020-11-21 22:13:54 +01:00
Alexander Weiss
c3ddde9e7d
Return hdrSize in ListPack
2020-11-21 22:13:54 +01:00
rawtaz
cac481634c
Merge pull request #3115 from johanbove/patch-1
...
Docs: Update 04_backup.rst
2020-11-20 11:23:03 +01:00
Johan Bové
c23b1a4cba
Update 04_backup.rst
...
Fixed typo - _files_ are included from _folders_, not other _files_.
2020-11-20 07:52:23 +01:00
Alexander Neumann
110a32a08b
Merge pull request #3113 from aawsome/fix-prune-stats-duplicates
...
Fix statistics in prune for duplicates
2020-11-19 20:34:11 +01:00
greatroar
8e213e82fc
backend/local: replace fs.Walk with custom walker
...
This code is more strict in what it expects to find in the backend:
depending on the layout, either a directory full of files or a directory
full of such directories.
2020-11-19 16:46:42 +01:00
Alexander Neumann
8a150ee91f
Merge pull request #3112 from MichaelEischer/reduce-progress-log-spam
...
Limit progress bar updates to once per second on non-terminal outputs
2020-11-19 07:54:36 +01:00
Alexander Weiss
cb5ec7ea6b
Fix statistics in prune for duplicates
...
Note that this fix only solves the statistics problem, if
all duplicates are marked for repacking.
If not all duplicates are marked for repacking, we lack the
information which
The situation that not all duplicates are marked for repacking can occur
when using the `max-repack-size` option
2020-11-18 22:30:22 +01:00
Michael Eischer
625410f003
Limit progress bar updates to once per second on non-terminal outputs
...
The code accidentally checked whether stdin is a terminal instead of
stdout, the former is not relevant here as the output is printed on
stdout.
2020-11-18 22:12:07 +01:00
Alexander Neumann
75eff92b56
Merge pull request #3107 from eleith/do-not-require-bucket-permissions-for-init
...
do not require gs bucket permissions to init repository
2020-11-18 16:53:45 +01:00
eleith
a24e986b2b
do not require gs bucket permissions to init repository
...
a gs service account may only have object permissions on an existing
bucket but no bucket create/get permissions.
these service accounts currently are blocked from initialization a
restic repository because restic can not determine if the bucket exists.
this PR updates the logic to assume the bucket exists when the bucket
attribute request results in a permissions denied error.
this way, restic can still initialize a repository if the service
account does have object permissions
fixes: https://github.com/restic/restic/issues/3100
2020-11-18 06:14:11 -08:00
rawtaz
6822ce8479
Merge pull request #3105 from restic/fix-init-repo-file
...
Allow using --repository-file in init
2020-11-17 21:54:44 +01:00
Alexander Neumann
d857fb6e59
Allow using --repository-file in init
2020-11-17 20:43:46 +01:00
rawtaz
342520b648
Merge pull request #3103 from greatroar/rephrase-pr3102
...
Rephrase #3095/#3102 changelog entry
2020-11-17 12:52:12 +01:00
greatroar
028f2b8c0e
Rephrase #3095/#3102 changelog entry
2020-11-17 12:37:06 +01:00
rawtaz
1b6e8c888f
Merge pull request #3102 from tofran/remove-rclone-drive-use-tras-default-param
...
Remove `--drive-use-trash=false` from rclone param
2020-11-17 11:21:20 +01:00
Alexander Neumann
5f3b802ee7
Merge pull request #3099 from MichaelEischer/check-less-memory
2020-11-16 10:11:25 +01:00
Michael Eischer
022dc35be9
Add changelog
2020-11-15 19:02:51 +01:00
Michael Eischer
1f43cac12d
check: Only track data blobs when unused blobs should be reported
...
This improves the memory usage of check a lot as it now only has to
track tree blobs when run using the default parameters.
2020-11-15 18:43:07 +01:00
Michael Eischer
6da66c15d8
check: Simplify referenced blob tracking
...
The result is identical as long as the context in not canceled. However,
in that case the result is incomplete anyways.
2020-11-15 18:42:55 +01:00
Michael Eischer
3500f9490c
check: Simplify blob status tracking
...
UnusedBlobs now directly reads the list of existing blobs from the
repository index. This removes the need for the blobStatusExists flag,
which in turn allows converting the blobRefs map into a BlobSet.
2020-11-15 18:42:42 +01:00
Michael Eischer
b8c7543a55
check: Merge 'size could not be found' and 'not found in index' errors
...
By construction these two errors always show up in pairs: 'size could
not be found' is printed when the blob is not found in the repository
index. That blob is also part of the `blobs` array. Later on, check
iterates over that array and checks whether the blob is marked as
existing. Which cannot be the case as that mark is generated by
iterating over the repository index.
The merged warning no longer reports the blob index within a file. That
information could also be derived by printing the affected tree using
`cat` and searching for the blob.
2020-11-15 18:41:50 +01:00
MichaelEischer
45ba456291
Merge pull request #3038 from fgma/check-data-subset-percentage
...
Check data subset: check random percentage subset
2020-11-15 18:24:13 +01:00
Michael Eischer
caac38ed27
check: Tweak subset percentage over 100% error message
2020-11-15 18:13:50 +01:00
Michael Eischer
15c537f9db
Rename changelog entry to contain issue id
2020-11-15 18:13:50 +01:00
fgma
8f9cea8cc0
Check data subset: check random percentage subset
2020-11-15 18:13:50 +01:00
Alexander Neumann
3c0c0c132b
Merge pull request #3006 from aawsome/new-rebuild-index
...
Reimplement rebuild-index and remove /internal/index
2020-11-15 17:48:43 +01:00
Alexander Neumann
9968220652
Merge pull request #2850 from greatroar/packer-malloc
...
Decrease allocation rate in internal/pack
2020-11-15 17:19:49 +01:00
kitone
0649828555
Add changelog
2020-11-15 17:09:30 +01:00
kitone
3c03b35212
Fix #1681 should not try to create the mount point if it doesn't exist, rather return an error
2020-11-15 17:09:30 +01:00
greatroar
ab2b7d7f9a
Decrease allocation rate in internal/pack
...
internal/repository benchmark results:
name old time/op new time/op delta
PackerManager-8 179ms ± 1% 181ms ± 1% +0.78% (p=0.009 n=10+10)
name old speed new speed delta
PackerManager-8 294MB/s ± 1% 292MB/s ± 1% -0.77% (p=0.009 n=10+10)
name old alloc/op new alloc/op delta
PackerManager-8 91.3kB ± 0% 72.2kB ± 0% -20.92% (p=0.000 n=9+7)
name old allocs/op new allocs/op delta
PackerManager-8 1.38k ± 0% 0.76k ± 0% -45.20% (p=0.000 n=10+7)
2020-11-15 16:51:47 +01:00
greatroar
9a8a2cae4c
Move pack testing logic to test file
2020-11-15 16:45:49 +01:00
Alexander Weiss
9607cad267
Remove internal/index
2020-11-15 07:05:09 +01:00
Alexander Weiss
3d1d5295cc
Add changelog
2020-11-15 07:05:09 +01:00
Alexander Weiss
30b6a0878a
Reimplement rebuild-index
2020-11-15 07:05:09 +01:00
Alexander Weiss
187c8fb259
Parallelize MasterIndex.Save()
2020-11-15 07:05:09 +01:00
Alexander Weiss
1ec628ddf5
Add extraObsolete to MasterIndex.Save
2020-11-15 07:05:09 +01:00
Alexander Weiss
5898cb341f
Use CreateIndexFromPacks() in test
2020-11-15 07:05:05 +01:00
Alexander Weiss
43732bb885
Add CreateIndexFromPacks()
2020-11-15 07:04:51 +01:00
MichaelEischer
6feaf6bd1f
Merge pull request #3030 from greatroar/concrete-statt
...
Replace restic.statT interface by concrete types
2020-11-14 23:32:17 +01:00
greatroar
c45f8ee075
Replace restic.statT interface by concrete types
...
name old time/op new time/op delta
NodeFillUser-8 1.81µs ± 9% 1.50µs ± 5% -17.07% (p=0.000 n=19+20)
NodeFromFileInfo-8 1.76µs ± 4% 1.49µs ± 6% -15.63% (p=0.000 n=20+19)
name old alloc/op new alloc/op delta
NodeFillUser-8 496B ± 0% 352B ± 0% -29.03% (p=0.000 n=20+20)
NodeFromFileInfo-8 496B ± 0% 352B ± 0% -29.03% (p=0.000 n=20+20)
name old allocs/op new allocs/op delta
NodeFillUser-8 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.000 n=20+20)
NodeFromFileInfo-8 3.00 ± 0% 2.00 ± 0% -33.33% (p=0.000 n=20+20)
2020-11-14 23:23:26 +01:00
MichaelEischer
3601a9b6cd
Merge pull request #2690 from SkYNewZ/master
...
Fix #2688 : Handle comma-separated list tags when using backup command
2020-11-14 23:19:42 +01:00
Michael Eischer
fdec8051ab
tags: Tweak description of --add/set/remove options
2020-11-14 22:55:30 +01:00
MichaelEischer
333c5a19d4
Merge pull request #3082 from aawsome/check-sizes
...
Check: check sizes of packs from index and packheader
2020-11-14 22:37:42 +01:00
Quentin Lemaire
a8ad6b9a4b
fix(tags): Change tags list type according to restic.TagList
2020-11-14 16:24:58 +00:00
Quentin Lemaire
b0882b3f3c
fix(snapshots): Update help message to better match the 'forget' command one
2020-11-14 15:48:57 +00:00
Quentin Lemaire
e74110a833
docs: Write new entry to changelog/unreleased
2020-11-14 15:48:56 +00:00
Quentin Lemaire
ae441d3134
fix(backup): Switch tags cobra type to handle comma-separated list
2020-11-14 15:48:56 +00:00
Alexander Neumann
913a34f568
Merge pull request #3094 from restic/fix-ci-tests
...
Fix ci tests
2020-11-14 11:24:28 +01:00
Alexander Neumann
468612b108
Disable fuse test for macOS
2020-11-14 11:02:49 +01:00
Alexander Weiss
7eabcabf68
Adjust changelog
2020-11-14 00:42:49 +01:00
Alexander Weiss
17bb77b1f9
check: Also check blob length and offset
2020-11-14 00:42:49 +01:00
Alexander Weiss
80dcfca191
check: Check sizes computed from index and pack header
2020-11-14 00:42:49 +01:00
tofran
94a154c7ca
Remove `--drive-use-trash=false` from rclone param
...
Google drive trash retention policy changed making this
no longer a good default
a go
Issue #3095
2020-11-13 22:58:48 +00:00
Alexander Neumann
219d9a62f2
Download assets from repo
2020-11-13 21:56:14 +01:00
rawtaz
e8713bc209
Merge pull request #3093 from restic/fix-diff-metadata
...
diff: Correctly count top-level blobs
2020-11-13 21:50:29 +01:00
Alexander Neumann
04d1983800
Merge pull request #3090 from fgma/vss-fix-386
...
vss: fix DeleteSnapshots() and GetSnapshotProperties() on 386
2020-11-13 21:30:34 +01:00
fgma
88208c3db2
vss: improved changelog entry
2020-11-13 21:13:17 +01:00
Alexander Neumann
59ea5a4208
diff: Correctly count top-level blobs
2020-11-13 21:11:21 +01:00
Alexander Neumann
145830005b
Use old versions, otherwise tar doesn't work
2020-11-12 21:35:01 +01:00
Alexander Neumann
8ad9f88993
helpers: Improve error message
2020-11-12 20:38:31 +01:00
fgma
859d89b032
vss: add changelog file for issue 3090
2020-11-12 19:38:22 +01:00
fgma
f9223cd827
vss: fix DeleteSnapshots() and GetSnapshotProperties() on 386
2020-11-12 19:31:00 +01:00
Alexander Neumann
0be906a92f
CI: Use netcologne mirror
2020-11-11 20:59:23 +01:00
rawtaz
dfb9326b1b
Merge pull request #3085 from LordGaav/s3-list-objects-v1-flag
...
Extended option to select V1 API for ListObjects on S3 backend
2020-11-11 20:52:44 +01:00
Alexander Neumann
e4e0ce09ad
CI: Update links for Windows binaries
2020-11-11 20:51:01 +01:00
Alexander Neumann
0334114865
CI: Hardcode mirror URL
2020-11-11 20:42:47 +01:00
Alexander Neumann
b1bbdcb637
Improve changelog
2020-11-11 20:30:30 +01:00
Alexander Neumann
4a0b7328ec
s3: Remove dots for config description
2020-11-11 20:20:35 +01:00
Alexander Neumann
3e0456d88b
Highlight that s3.list-objects-v1 is a temporary
2020-11-11 20:11:35 +01:00
Alexander Neumann
dd94174379
Merge pull request #3086 from greatroar/diff-errors
...
Improve error reporting from restic diff
2020-11-11 19:48:03 +01:00
greatroar
63e32c44c0
Improve error reporting from restic diff
...
Instead of a stacktrace, restic diff 111 222 now reports:
Fatal: no matching ID found for prefix "111"
2020-11-11 16:40:40 +01:00
Nick Douma
f013662e3f
Remove separate section on Ceph, and move s3.list-objects-v1 note to S3 section
2020-11-11 15:11:14 +01:00
Nick Douma
4320ff2bbf
Add changelog entry for s3.list-objects-v1
2020-11-11 12:33:22 +01:00
Nick Douma
354b7e89cc
Document the extended s3.list-objects-v1 flag in a new Ceph section
2020-11-11 12:32:46 +01:00
Nick Douma
829959390a
Provide UseV1 parameter to minio.ListObjectsOptions based on s3.list-objects-v1
2020-11-11 11:54:38 +01:00
Nick Douma
ccd55d529d
Add s3.list-objects-v1 extended option and default to false
2020-11-11 11:54:36 +01:00
Nick Douma
4ddcc17135
Add support for boolean extended options
2020-11-11 11:54:27 +01:00
MichaelEischer
407843c5f9
Merge pull request #3034 from hoyho/gh_master
...
bugfix: omit ENOTDATA for extended attributes
2020-11-09 22:38:49 +01:00
MichaelEischer
46d31ab86d
Merge pull request #3058 from greatroar/counter
...
Replace restic.Progress with new progress.Counter (fixes two race conditions)
2020-11-09 22:19:09 +01:00
Alexander Neumann
c986823d3f
Merge pull request #3048 from aawsome/check-pack-index
...
check: check index for packs that are read
2020-11-09 20:12:32 +01:00
Alexander Weiss
239931578c
check: check index for packs that are read
2020-11-09 17:28:14 +01:00
hoyho
9df52327cc
bugfix: omit ENOTDATA for extended attributes
...
Signed-off-by: hoyho <luohaihao@gmail.com>
2020-11-10 00:20:34 +08:00
greatroar
21b787a4d1
Stop Counters where they're constructed and started
2020-11-09 13:03:31 +01:00
greatroar
ddca699cd2
Replace restic.Progress with new progress.Counter
...
This fixes two race conditions while cleaning up the code.
2020-11-09 12:12:35 +01:00
rawtaz
605db3b389
Merge pull request #3077 from MichaelEischer/tame-golangci-lint
...
Switch to the default set of golangci-lint linters
2020-11-08 22:34:27 +01:00
Michael Eischer
8de129e12f
Switch to the default set of golangci-lint linters
2020-11-08 22:17:50 +01:00
Alexander Neumann
2072f0a481
Merge pull request #3076 from restic/check-go-mod-tidy
...
Check that go.mod/go.sum are up to date
2020-11-08 21:37:22 +01:00
Alexander Neumann
5731e391f8
Merge pull request #3075 from restic/warn-prune-no-cache
...
UI: Add several hints
2020-11-08 20:37:10 +01:00
Alexander Neumann
6a0a1d1f1c
Check that go.mod/go.sum are up to date
2020-11-08 20:32:25 +01:00
Alexander Neumann
a8d21b5dcf
prune: Warn if no cache is present
2020-11-08 20:25:35 +01:00
Alexander Neumann
823d0afd6e
backup: Always print parent snapshot info
2020-11-08 20:25:35 +01:00
Alexander Neumann
a5989707ac
Move test for golang-ci
2020-11-08 19:58:16 +01:00
Alexander Neumann
3a0cfafeb5
Fix quotes
2020-11-08 19:52:37 +01:00
Alexander Neumann
c923bd957d
Fix test for pull request
2020-11-08 19:51:11 +01:00
Alexander Neumann
1a3f885d3d
Only run golangci-lint for PRs
2020-11-08 17:34:07 +01:00
Alexander Neumann
3bf43d7951
Merge pull request #2475 from restic/use-github-actions
...
Use GitHub Actions for CI
2020-11-08 17:30:12 +01:00
Alexander Neumann
561da92396
Replace badges in README
2020-11-08 17:26:59 +01:00
Alexander Neumann
5cf42884c8
Run tests on GitHub Actions
2020-11-08 16:04:57 +01:00
MichaelEischer
9e4e0077fb
Merge pull request #3069 from greatroar/xattr-eintr
...
Update pkg/xattr to handle EINTR on Linux
2020-11-08 12:52:23 +01:00
MichaelEischer
1758da855f
Merge pull request #3068 from aawsome/context-mem-backend
...
Return context error in mem backend
2020-11-08 12:24:12 +01:00
greatroar
15ea90feed
Update pkg/xattr to handle EINTR on Linux
...
Updates #2659 . This is a case where the stdlib will not handle EINTR for
us, even with Go 1.16. That xattr calls are directly affected can be
seen in the report for issue #2968 .
2020-11-08 09:34:24 +01:00
Alexander Weiss
826cfa0533
fix context in archiver tests
2020-11-08 08:24:24 +01:00
Alexander Weiss
fef408a8bd
Return context error in mem backend
2020-11-08 00:05:53 +01:00
greatroar
a2d4209322
Don't recurse in local backend's List if not required
...
Due to the return if !isFile, the IsDir branch in List was never taken
and subdirectories were traversed recursively.
Also replaced isFile by an IsRegular check, which has been equivalent
since Go 1.12 (golang/go@a2a3dd00c9 ).
2020-11-07 08:54:13 +01:00
Alexander Neumann
275f713211
Remove Travis and AppVeyor
2020-11-06 21:49:52 +01:00
MichaelEischer
4707bdb204
Merge pull request #2842 from aawsome/rebuild-index-inmem
...
Rebuild index in prune by using in-memory index
2020-11-06 20:51:20 +01:00
Alexander Neumann
47277c4b4c
Add comments, clarify computation
2020-11-06 20:23:30 +01:00
Alexander Weiss
d2e53730d6
Add test that repo.List is only called once
2020-11-06 20:23:30 +01:00
Alexander Weiss
fd33030556
Use in-memory index to rebuild index in prune
2020-11-06 20:23:30 +01:00
Alexander Weiss
38cc4393f6
Add Masterindex.Save(); Add Index.Packs()
2020-11-06 20:23:30 +01:00
Alexander Neumann
7f6f31c34b
Remove Hound
2020-11-06 11:43:58 +01:00
Alexander Neumann
164b4cb2f6
Add changelog for #3014
2020-11-06 10:05:42 +01:00
Alexander Neumann
4a9b05aff1
Merge pull request #3063 from aawsome/fix-3062
...
Fix #3062
2020-11-05 19:36:40 +01:00
Alexander Weiss
aaf1c44362
Fix #3062
2020-11-05 17:05:42 +01:00
Alexander Neumann
a5592e83f7
Merge pull request #3014 from ivandeex/pr-stream-reset
...
Fix sporadic stream reset between rclone and restic
2020-11-05 16:11:50 +01:00
Ivan Andreev
ab2790d9de
Fix http2 stream reset between restic and rest backends #3014
2020-11-05 15:57:40 +03:00
Alexander Neumann
8a2a326189
Merge pull request #2535 from ncw/fix-2528
...
s3: add bucket-lookup parameter to select path or dns style bucket lookup
2020-11-05 12:39:34 +01:00
Nick Craig-Wood
86b5d8ffaa
s3: add bucket-lookup parameter to select path or dns style bucket lookup
...
This is to enable restic working with Alibaba cloud
Fixes #2528
2020-11-05 12:20:10 +01:00
Alexander Neumann
636b2f2e94
Merge pull request #2941 from MichaelEischer/parallel-repack
...
prune: Parallelize repack step
2020-11-05 11:00:41 +01:00
Alexander Neumann
ae5302c7a8
Add comment that keepBlobs is modified
2020-11-05 10:33:38 +01:00
Alexander Neumann
866a52ad4e
Remove unneeded seek
...
The file returned from DownloadAndHash() is already seeked to the start
of the file.
2020-11-05 10:31:49 +01:00
Alexander Neumann
a4507610a0
Fix typo
2020-11-05 10:31:49 +01:00
Alexander Neumann
7def2d8ea7
Use a non-constant seed
2020-11-05 10:31:49 +01:00
Alexander Neumann
ee0112ab3b
Clarify message about expected error
2020-11-05 10:31:49 +01:00
Michael Eischer
b373f164fe
prune: Parallelize repack command
2020-11-05 10:31:49 +01:00
Alexander Neumann
8a0dbe7c1a
helpers: Remove old changelog files for release
2020-11-05 10:26:00 +01:00
Alexander Neumann
4e3ad8b3b1
Remove changelog files from 0.11.0 release
2020-11-05 10:22:39 +01:00
Alexander Neumann
5144141321
Merge pull request #2718 from aawsome/new-cleanup-command
...
Reimplementation of prune
2020-11-05 10:12:19 +01:00
Alexander Neumann
d35d279455
Set development version for 0.11.0
2020-11-05 09:41:40 +01:00
Alexander Neumann
b7e1ece1e0
Add version for 0.11.0
2020-11-05 09:41:35 +01:00
Alexander Neumann
c5300a2c56
Update manpages and auto-completion
2020-11-05 09:41:35 +01:00
Alexander Neumann
1bd92896d7
Generate CHANGELOG.md for 0.11.0
2020-11-05 09:41:22 +01:00
Alexander Neumann
9dba01021e
Prepare changelog for 0.11.0
2020-11-05 09:41:22 +01:00
Alexander Neumann
23f6b8c3fd
Add .vscode to gitignore
2020-11-05 09:40:56 +01:00
Alexander Neumann
79a50e3b1f
helpers: Create changelog subdir for release
2020-11-05 09:39:06 +01:00
Alexander Neumann
b8a5ca2d10
Add gitignore to changelog/unreleased
2020-11-05 09:01:32 +01:00
fgma
916b2d303b
vss: fix potential crash (not reachable in restic) ( #3045 )
...
HasSufficientPrivilegesForVSS() now returns an error
2020-11-04 22:14:18 +01:00
Alexander Neumann
a06f5c28c0
Merge pull request #3055 from greatroar/cleanup
...
Don't separately allocate sync.{Mutex,Once} if not necessary
2020-11-04 11:11:21 +01:00
greatroar
c2f3eee5af
Don't separately allocate sync.{Mutex,Once} if not necessary
...
Separate allocation of synchronization devices suggests they're shared
between objects, but they're not.
2020-11-04 09:55:43 +01:00
rawtaz
62345abe4a
Merge pull request #3053 from maikell/patch-1
...
Update README.md
2020-11-03 19:06:59 +01:00
Maikel van Leeuwen
e024fc6d4d
Update README.md
...
Typo in link.
Links to https://forum.restic.net%3E/ should be https://forum.restic.net
2020-11-03 18:51:11 +01:00
Alexander Neumann
1ca60bccfb
Refactor condition for MaxRepackBytes
...
Don't depend on the string (opts.MaxRepackSize) for the condition,
instead check if there's a (positive) limit configured.
2020-11-03 16:42:21 +01:00
Alexander Neumann
7f86eb4ec0
Move helper function
2020-11-03 16:42:21 +01:00
Alexander Neumann
c1a3de4a6e
Refactor max-unused calculation, add `unlimited` option
...
Add a callback to the PruneOptions struct which calculates the number of
bytes allowed to be unused after prune is done. This way, the logic is
closer to the option parsing code.
Also, add an explicit option `unlimited` for the use case when storage
does not matter but bandwidth and time do. Internally, this sets the
maximum number of unused bytes to MaxUint64.
Rework the documentation slightly so that no more "packs" are
mentioned and it talks about "files" instead.
Make it clear in the documentation that the percentage given to
`--max-unused` is relative to the whole repository size after pruning is
done. If specified, it must be below 100%, otherwise the repository
would contain 100% of unused data, which is pointless.
I had a hard time coming up with the correct formula to calculate the
maximum number of unused bytes based on the number of used bytes. For a
fraction `p` (0 ≤ p < 1), a repo with `u` bytes used, and the number of
unused bytes `x` the following holds:
x ≤ p * (u+x)
⇔ x ≤ p*u + p*x
⇔ x - p*x ≤ p*u
⇔ x * (1-p) ≤ p*u
⇔ x ≤ p/(1-p) * u
2020-11-03 16:42:21 +01:00
Alexander Neumann
f8c4dd7b1a
Split packe rewrite logic into two case branches
...
The comma is too sublte, let's split this into two separate branches.
2020-11-03 16:42:21 +01:00
Alexander Neumann
a5b80452fe
Add comment that usedBlobs is modified
2020-11-03 16:42:21 +01:00
Alexander Neumann
aff1e220f5
Split struct members, add comments
2020-11-03 16:42:21 +01:00
Alexander Neumann
095155d9ce
Remove RepackSmall
2020-11-03 16:42:21 +01:00
Alexander Neumann
1dd9fdce74
Reword changelog slightly
2020-11-03 16:42:21 +01:00
Alexander Weiss
b2f5381737
Make realistic forget --prune --dryrun
2020-11-03 16:42:21 +01:00
Alexander Weiss
7f9a0a5907
Reimplementation of prune
2020-11-03 16:42:21 +01:00
Alexander Weiss
3b591ed987
Add Verboseff
2020-11-03 16:42:21 +01:00
Alexander Weiss
ce7d613749
Add Blob.Handle()
2020-11-03 16:42:21 +01:00
Alexander Weiss
581d90cf91
Make some pack parameters public
2020-11-03 16:42:21 +01:00
Alexander Neumann
0db9024aad
Merge pull request #3052 from restic/error-on-invalid-size
...
Return an error for invalid sizes
2020-11-03 12:13:49 +01:00
Alexander Neumann
21ba15577e
Return an error for invalid sizes
2020-11-03 11:53:49 +01:00
Alexander Neumann
2fb1957ca4
Rewrite README as Markdown
2020-11-03 09:20:09 +01:00
Alexander Neumann
9a88fb253b
Merge pull request #3051 from greatroar/sanitize-env
...
Sanitize environment before starting backend processes (rclone, ssh)
2020-11-02 21:18:57 +01:00
Alexander Neumann
f14436953a
Merge pull request #3050 from greatroar/widechars
...
Fix string truncation in ui/termstatus
2020-11-02 16:58:51 +01:00
greatroar
11fbaaae9a
Sanitize environment before starting backend processes (rclone, ssh)
...
The restic security model includes full trust of the local machine, so
this should not fix any actual security problems, but it's better to be
safe than sorry.
Fixes #2192 .
2020-11-02 16:41:23 +01:00
greatroar
3ed84ff0c6
Fix string truncation in ui/termstatus
...
Fixes #3046 .
2020-11-02 12:50:49 +01:00
Alexander Neumann
8e965ed4eb
Merge pull request #3039 from MichaelEischer/tar-proper-dirs
...
Properly dump directories to tar
2020-11-02 12:30:26 +01:00
Alexander Neumann
5f0fa2129e
Improve readability
...
It's time to use a switch statement.
2020-11-02 11:24:26 +01:00
Alexander Neumann
04dfa19c7e
Improve changelog
2020-11-02 11:23:09 +01:00
Alexander Neumann
6509c207f4
Merge pull request #2997 from MichaelEischer/faster-excludes
...
Speedup exclude/include checking
2020-11-02 11:07:21 +01:00
Alexander Neumann
445b845267
Merge pull request #2978 from MichaelEischer/warn-tree-error
...
Warn if backup failed to read tree blob
2020-11-02 10:14:12 +01:00
Alexander Neumann
3ff37215df
Merge pull request #2935 from MichaelEischer/upgrade-minio
...
Upgrade minio SDK to version 7
2020-11-02 09:09:10 +01:00
MichaelEischer
5d379b5359
Merge pull request #3042 from rubiojr/remove-unused-function
...
Remove unused function
2020-11-01 21:55:22 +01:00
Sergio Rubio
e708628cfd
Remove unused function
...
Not currently used, and it'd need to be added to the MasterIndex interface first.
2020-10-28 13:24:49 +01:00
MichaelEischer
bb4b3481a6
Merge pull request #3031 from greatroar/debug-no-seek
...
Allow debug logging to pipes and terminals
2020-10-25 17:59:06 +01:00
MichaelEischer
ad3a52e6f0
Merge pull request #3026 from greatroar/refactor-ui
...
internal/ui refactoring
2020-10-25 17:51:39 +01:00
rawtaz
e8b4d8d8bc
Merge pull request #2998 from greatroar/env-docs
...
Systematize documentation of environment variables
2020-10-24 22:49:31 +02:00
Michael Eischer
1aa61e6def
Add changelog
2020-10-24 22:49:29 +02:00
Michael Eischer
8d7d6ad2d5
dump: include username in tar
2020-10-24 22:42:01 +02:00
Michael Eischer
fe09e6f865
dump: test proper permissions and directory name
2020-10-24 22:42:01 +02:00
Michael Eischer
1e3c9a2c11
dump: Fix file permission to tar mapping
...
The file permissions included a go specific directory bit which
accidentially forced the usage of the GNU header format. This leads
to problems with 7zip on Windows or when extended attributes are
used.
2020-10-24 22:42:01 +02:00
Michael Eischer
e21dcb0eea
dump: Additional ACL tests
2020-10-24 22:41:01 +02:00
MichaelEischer
31b8d7a639
Merge pull request #2274 from fgma/master
...
Support for Volume Shadow Copy Service (VSS) on windows
2020-10-24 15:43:34 +02:00
fgma
5695f9ebd2
vss: Implement VSS support for Windows
...
The VSS support works for 32 and 64-bit windows, this includes a check that
the restic version matches the OS architecture as required by VSS. The backup
operation will fail the user has not sufficient permissions to use VSS.
Snapshotting volumes also covers mountpoints but skips UNC paths.
2020-10-24 11:35:57 +02:00
greatroar
8091151638
doc: Update restic mount availability and requirements
...
NetBSD doesn't support restic mount either, so it's easier to list the
positive cases. Also noted that FUSE for macOS is required on the Mac.
2020-10-22 09:57:30 +02:00
greatroar
ae179ee63e
Systematize documentation of environment variables
...
Cache locations were documented inconsistently in three places.
The backup docs mentioned PATH being used to find fusermount, which is
never run by restic backup. It now mentions ssh and rclone, which are
used by backends.
The notion of a "system-wide" environment variable makes no sense.
TMPDIR is now mentioned because it allows for optimization and may
have security implications.
2020-10-22 09:57:30 +02:00
MichaelEischer
0590e3e12d
Merge pull request #3035 from greatroar/no-unmount-on-failure
...
restic mount: don't umount unless we actually tried to mount
2020-10-21 22:10:42 +02:00
greatroar
3807d13bdc
restic mount: don't umount unless we actually tried to mount
...
Previously, failure to open a repository would result in an "unable to
umount" message.
2020-10-21 12:15:05 +02:00
greatroar
63be3704d9
Allow debug logging to pipes and terminals
2020-10-19 22:30:30 +02:00
greatroar
35419de232
Simplify ui.StdioWrapper.Write
...
Instead of looping to find line breaks, make it look for the last one.
2020-10-17 20:30:46 +02:00
greatroar
863a590a81
Refactor termstatus.Term.{Print,Error} methods
2020-10-17 20:30:46 +02:00
greatroar
7c0b6a82db
Remove unused public method ui.linesWriter.Flush
2020-10-17 20:30:24 +02:00
MichaelEischer
96a912b65a
Merge pull request #3025 from aawsome/fix-index-ids
...
Fix setting of ID in DecodeIndex
2020-10-17 09:54:45 +02:00
Alexander Weiss
b44ecde8b0
Fix setting of ID in DecodeIndex
2020-10-17 09:12:58 +02:00
MichaelEischer
39fe1e96fe
Merge pull request #3018 from greatroar/background-check
...
Fix IsProcessBackground on Linux with stdin redirection
2020-10-16 22:50:19 +02:00
MichaelEischer
4ba237bb93
Merge pull request #3019 from greatroar/refactor-decodeindex
...
Refactor index decoding
2020-10-15 23:22:33 +02:00
MichaelEischer
ce87fbd7dc
Merge pull request #3022 from greatroar/refactor-runworkers
...
Defer channel closing outside repository.RunWorkers
2020-10-14 22:44:42 +02:00
greatroar
b27375f5ce
defer close(ch) outside repository.RunWorkers
2020-10-14 15:50:16 +02:00
greatroar
720e0ee0c7
if cond { return true }; return false => return cond
2020-10-13 20:56:43 +02:00
greatroar
27db3ec262
Refactor index decoding
...
Decoding old-format indices no longer requires loading and decrypting
twice.
2020-10-13 20:47:50 +02:00
greatroar
f80b07b2c8
Fix IsProcessBackground on Linux with stdin redirection
...
The previous implementation assumed that stdin was a terminal.
It now checks the terminal's fd.
2020-10-13 13:12:02 +02:00
MichaelEischer
6003dada14
Merge pull request #3011 from restic/doc-alpine
...
doc: Add restic install instructions for Alpine Linux
2020-10-12 23:29:40 +02:00
MichaelEischer
41a45ae908
Merge pull request #3016 from greatroar/uid-gid-lookup
...
Simplify/optimize cached UID/GID lookups
2020-10-12 23:24:55 +02:00
MichaelEischer
8299c5559c
Merge pull request #3015 from greatroar/changelog-template
...
Improved changelog template
2020-10-12 23:12:29 +02:00
Alexander Neumann
56883817d8
Merge pull request #2990 from MichaelEischer/fix-goreport-warnings
...
Fix some goreport warnings
2020-10-12 20:44:56 +02:00
greatroar
f0cd16e5ea
Cache uint32-typed ids in lookup{Username,Group}
...
NodeFillUser-8 1.92µs ± 5% 1.75µs ± 3% -8.89% (p=0.000 n=10+10)
NodeFromFileInfo-8 1.89µs ± 7% 1.76µs ± 4% -6.69% (p=0.001 n=10+10)
name old alloc/op new alloc/op delta
NodeFillUser-8 504B ± 0% 496B ± 0% -1.59% (p=0.000 n=10+10)
NodeFromFileInfo-8 504B ± 0% 496B ± 0% -1.59% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
NodeFillUser-8 5.00 ± 0% 3.00 ± 0% -40.00% (p=0.000 n=10+10)
NodeFromFileInfo-8 5.00 ± 0% 3.00 ± 0% -40.00% (p=0.000 n=10+10)
2020-10-12 15:08:15 +02:00
greatroar
a03fe4562e
Remove unused error return in lookup{Username,Group}
2020-10-12 14:48:23 +02:00
greatroar
efc075df87
Improved changelog template
2020-10-12 12:15:15 +02:00
Leo R. Lundgren
f500b0d90e
doc: Add restic install instructions for Alpine Linux
2020-10-11 18:31:07 +02:00
Alexander Neumann
c193cea119
Pass Context to NewRestorer()
2020-10-10 15:24:26 +02:00
Alexander Neumann
f0d49ca600
Merge pull request #2933 from MichaelEischer/less-context-todo
...
Replace most usages of context.TODO()
2020-10-10 15:03:44 +02:00
MichaelEischer
164d8af3dd
Merge pull request #2906 from kitone/fix-inconsistent-timestamps-permissions
...
Restore inconsistent timestamps permissions
2020-10-10 14:56:43 +02:00
kitone
052564007a
Add changelog
2020-10-10 14:37:51 +02:00
kitone
6099f81692
Fix #1212 restore code produces inconsistent timestamps/permissions.
...
Keep track of restored child status so parent and root directory not selected by filter will also restore metadata when traversing tree.
2020-10-10 13:46:44 +02:00
kitone
9d7f616190
Improve restorer debug log information
2020-10-10 13:46:43 +02:00
kitone
295ddb9e57
Add test case for inconsistent timestamps and permissions restoration
...
Reproduce from https://github.com/restic/restic/issues/1212
2020-10-10 13:46:43 +02:00
Michael Eischer
e638b46a13
Embed context into ReaderAt
...
The io.Reader interface does not support contexts, such that it is
necessary to embed the context into the backendReaderAt struct. This has
the problem that a reader might suddenly stop working when it's
contained context is canceled. However, this is now problem here as the
reader instances never escape the calling function.
2020-10-09 22:39:07 +02:00
Michael Eischer
d6cfe857b7
pass proper context into MasterIndex.RebuildIndex
2020-10-09 22:39:07 +02:00
Michael Eischer
2964d2ad15
Skip unlocking for nil locks
...
Now that lockRepo receives a context, it is possible that it is canceled
before a lock was created. Thus `unlockRepo` must be able to handle this
case.
2020-10-09 22:39:06 +02:00
Michael Eischer
0c9efa9c2a
Pass context to lockRepo
2020-10-09 22:39:06 +02:00
Michael Eischer
37a5e2d681
rest: use global context on repository creation
2020-10-09 22:39:06 +02:00
Michael Eischer
4b0fcaed45
unlock: use proper context for locks cleanup
...
The list operation used by RemoveStaleLocks or RemoveAllLocks will
already be canceled by the passed in context. Therefore we can also just
cancel the remove operation as the unlock command won't process all lock
files anyways.
2020-10-09 22:37:56 +02:00
Michael Eischer
645a6efaf2
restorer: remove redundant type specification
2020-10-09 22:37:56 +02:00
Michael Eischer
dc2e664209
integration_fuse_test: use global context
...
No need to use the TODO context.
2020-10-09 22:37:56 +02:00
Michael Eischer
a449450021
init: pass proper context to master key generation
...
This is no change in behavior as a canceled context did later on cause
the config file creation to fail. Therefore this change just lets the
repository initialization fail a bit earlier.
2020-10-09 22:37:56 +02:00
Michael Eischer
603bb0e309
restore: Use proper context while loading snapshot
2020-10-09 22:37:56 +02:00
Michael Eischer
27456f6545
debug: use proper context
...
This allows the debug commands to be properly interrupted.
2020-10-09 22:37:56 +02:00
Michael Eischer
c458e114d4
pass context to Find / FindSnapshot
...
This allows proper interruption of restic while it searches for
snapshots or key files.
2020-10-09 22:37:56 +02:00
Michael Eischer
45e9a55c62
Wire context into backend layout detection
2020-10-09 22:37:24 +02:00
Michael Eischer
307a6ba3a3
Upgrade minio sdk to v7
...
This changes are primarily straightforward modifications to pass the
parameters in the now expected way.
2020-10-09 22:37:24 +02:00
Michael Eischer
50da20d93d
Warn if backup failed to read tree blob
2020-10-09 22:36:27 +02:00
Alexander Neumann
5fd3dbccb7
Merge pull request #2963 from MichaelEischer/fix-status-deadlock
...
backup: Fix possible deadlock of scanner goroutine
2020-10-09 21:35:50 +02:00
Alexander Neumann
35655a481b
Merge pull request #2931 from MichaelEischer/reduce-debug-overhead
...
Reduce debug log overhead
2020-10-09 21:35:11 +02:00
Alexander Neumann
30cb553c8d
Merge pull request #2932 from MichaelEischer/proper-rclone-create
...
Call rclone.Create to create a new repository for the rclone backend
2020-10-09 21:29:15 +02:00
Alexander Neumann
2f3eeff2e7
Merge pull request #2964 from MichaelEischer/fix-repro-instructions
...
Fix instructions to reproduce the release binaries
2020-10-09 21:28:24 +02:00
MichaelEischer
356ac404cd
Merge pull request #2996 from greatroar/setconsolecursorposition
...
Use windows.SetConsoleCursorPosition in ui/termstatus
2020-10-09 18:19:02 +02:00
rawtaz
ab769abeaf
Merge pull request #3002 from MichaelEischer/update-contributing
...
Update links to help wanted labels in CONTRIBUTING.md and link to forum
2020-10-09 17:24:25 +02:00
Michael Eischer
4036991c91
Update links to help wanted labels in CONTRIBUTING.md and link to forum
2020-10-09 16:15:43 +02:00
Michael Eischer
88c8e903d2
filter: Fix glob matching on absolute path marker on windows
...
A pattern part containing "/" is used to mark a path or a pattern as
absolute. However, on Windows the path separator is "\" such that glob
patterns like "?" could match the marker. The code now explicitly skips
the marker when the pattern does not represent an absolute path.
2020-10-09 16:11:05 +02:00
greatroar
740758a5fa
Optimize filter pattern matching
...
By replacing "**" with "", checking for this special path component can
be reduced to a length-zero check.
name old time/op new time/op delta
FilterLines-8 44.7ms ± 5% 44.9ms ± 5% ~ (p=0.631 n=10+10)
FilterPatterns/Relative-8 13.6ms ± 4% 13.4ms ± 5% ~ (p=0.165 n=10+10)
FilterPatterns/Absolute-8 10.9ms ± 5% 10.7ms ± 4% ~ (p=0.052 n=10+10)
FilterPatterns/Wildcard-8 53.7ms ± 5% 50.4ms ± 5% -6.00% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-8 128ms ± 2% 95ms ± 1% -25.54% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-8 3.57MB ± 0% 3.57MB ± 0% ~ (p=1.000 n=9+8)
FilterPatterns/Absolute-8 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.903 n=9+8)
FilterPatterns/Wildcard-8 19.7MB ± 0% 19.7MB ± 0% -0.00% (p=0.022 n=10+9)
FilterPatterns/ManyNoMatch-8 3.57MB ± 0% 3.57MB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-8 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-8 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-8 88.7k ± 0% 88.7k ± 0% ~ (all equal)
FilterPatterns/ManyNoMatch-8 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-09 15:51:14 +02:00
MichaelEischer
862ee4b2c9
Merge pull request #2970 from labkode/fopenskip
...
Skip fopen for file change check to avoid network penalty
2020-10-09 11:43:54 +02:00
Hugo Gonzalez Labrador
958dc6aafc
Skip fopen for file change check to avoid network penalty
2020-10-09 10:53:28 +02:00
MichaelEischer
88cc444779
Merge pull request #2934 from MichaelEischer/upgrade-backoff
...
Upgrade github.com/cenkalti/backoff module
2020-10-08 20:33:30 +02:00
MichaelEischer
4a424af1d5
Merge pull request #2989 from MichaelEischer/remove-local-chmod
...
local: mark repository files as read-only and handle chmod errors
2020-10-08 19:04:13 +02:00
Michael Eischer
bc65da2baf
Add changelog
2020-10-08 14:45:26 +02:00
Michael Eischer
b79f18209f
Upgrade github.com/cenkalti/backoff module
...
We now use v4 of the module. `backoff.WithMaxRetries` no longer repeats
an operation endlessly when a retry count of 0 is specified. This
required a few fixes for the tests.
2020-10-07 22:04:59 +02:00
Michael Eischer
8388e67c4b
filter: cleanup path separator conversion
2020-10-07 21:14:07 +02:00
Michael Eischer
0acc3c5923
filter: special case patterns without globbing characters
...
In case a part of a path is a simple string, we can just check for
equality without complex parsing in filepath.Match.
name old time/op new time/op delta
FilterLines-4 34.8ms ±17% 41.2ms ±23% +18.36% (p=0.000 n=10+10)
FilterPatterns/Relative-4 21.7ms ± 6% 12.1ms ±23% -44.46% (p=0.000 n=10+10)
FilterPatterns/Absolute-4 10.0ms ± 5% 9.1ms ±11% -9.80% (p=0.006 n=10+9)
FilterPatterns/Wildcard-4 47.0ms ± 7% 42.2ms ± 5% -10.19% (p=0.000 n=9+10)
FilterPatterns/ManyNoMatch-4 190ms ± 1% 131ms ±20% -31.47% (p=0.000 n=8+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.870 n=9+9)
FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.145 n=10+10)
FilterPatterns/Wildcard-4 14.3MB ± 0% 19.7MB ± 0% +37.91% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.421 n=10+9)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-4 88.7k ± 0% 88.7k ± 0% ~ (all equal)
FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-07 20:55:43 +02:00
Michael Eischer
54a124de3b
filter: explicitly test separate ListWithChild function
2020-10-07 20:47:52 +02:00
Michael Eischer
bcc3bddcf4
filter: only check whether a child path could match when necessary
...
When checking excludes there is no need to test whether a child path
could also match the pattern, as it is by definition excluded.
Previously childMayMatch was calculated but then discarded. For simple
absolute paths this can account for half the time spent for checking
pattern matches.
name old time/op new time/op delta
FilterPatterns/Relative-4 23.3ms ± 9% 21.7ms ± 6% -6.68% (p=0.004 n=10+10)
FilterPatterns/Absolute-4 13.9ms ± 7% 10.0ms ± 5% -27.61% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 51.4ms ± 7% 47.0ms ± 7% -8.51% (p=0.001 n=9+9)
FilterPatterns/ManyNoMatch-4 551ms ± 9% 190ms ± 1% -65.41% (p=0.000 n=10+8)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.665 n=10+9)
FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.480 n=9+10)
FilterPatterns/Wildcard-4 14.3MB ± 0% 14.3MB ± 0% ~ (p=0.431 n=9+10)
FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-4 88.7k ± 0% 88.7k ± 0% ~ (all equal)
FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-07 20:47:52 +02:00
Michael Eischer
17c53efb0d
filter: Optimize double wildcard expansion
...
This only allocates a single slice to expand the double wildcard and
only copies the pattern prefix once.
name old time/op new time/op delta
FilterPatterns/Relative-4 22.7ms ± 5% 23.3ms ± 9% ~ (p=0.353 n=10+10)
FilterPatterns/Absolute-4 14.2ms ±13% 13.9ms ± 7% ~ (p=0.853 n=10+10)
FilterPatterns/Wildcard-4 266ms ±16% 51ms ± 7% -80.67% (p=0.000 n=10+9)
FilterPatterns/ManyNoMatch-4 554ms ± 6% 551ms ± 9% ~ (p=0.436 n=10+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.349 n=10+10)
FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.073 n=10+9)
FilterPatterns/Wildcard-4 141MB ± 0% 14MB ± 0% -89.89% (p=0.000 n=10+9)
FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-4 1.63M ± 0% 0.09M ± 0% -94.56% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-07 20:47:48 +02:00
Michael Eischer
7959796269
filter: Special case for absolute paths
...
name old time/op new time/op delta
FilterPatterns/Relative-4 23.6ms ±20% 22.7ms ± 5% ~ (p=0.684 n=10+10)
FilterPatterns/Absolute-4 32.3ms ± 8% 14.2ms ±13% -56.01% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 334ms ±17% 266ms ±16% -20.56% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 709ms ± 7% 554ms ± 6% -21.89% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 3.57MB ± 0% 3.57MB ± 0% +0.00% (p=0.046 n=9+10)
FilterPatterns/Absolute-4 3.57MB ± 0% 3.57MB ± 0% ~ (p=0.464 n=10+10)
FilterPatterns/Wildcard-4 141MB ± 0% 141MB ± 0% ~ (p=0.163 n=9+10)
FilterPatterns/ManyNoMatch-4 3.57MB ± 0% 3.57MB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Absolute-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
FilterPatterns/Wildcard-4 1.63M ± 0% 1.63M ± 0% ~ (p=0.072 n=10+10)
FilterPatterns/ManyNoMatch-4 22.2k ± 0% 22.2k ± 0% ~ (all equal)
2020-10-07 20:47:29 +02:00
Michael Eischer
375c2a56de
filter: Parse filter patterns only once
...
name old time/op new time/op delta
FilterPatterns/Relative-4 30.3ms ±10% 23.6ms ±20% -22.12% (p=0.000 n=10+10)
FilterPatterns/Absolute-4 49.0ms ± 3% 32.3ms ± 8% -33.94% (p=0.000 n=8+10)
FilterPatterns/Wildcard-4 345ms ± 9% 334ms ±17% ~ (p=0.315 n=10+10)
FilterPatterns/ManyNoMatch-4 3.93s ± 2% 0.71s ± 7% -81.98% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 4.63MB ± 0% 3.57MB ± 0% -22.98% (p=0.000 n=9+9)
FilterPatterns/Absolute-4 8.54MB ± 0% 3.57MB ± 0% -58.20% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 146MB ± 0% 141MB ± 0% -2.93% (p=0.000 n=9+9)
FilterPatterns/ManyNoMatch-4 907MB ± 0% 4MB ± 0% -99.61% (p=0.000 n=9+9)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 66.6k ± 0% 22.2k ± 0% -66.67% (p=0.000 n=10+10)
FilterPatterns/Absolute-4 88.7k ± 0% 22.2k ± 0% -75.00% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 1.70M ± 0% 1.63M ± 0% -3.92% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 4.46M ± 0% 0.02M ± 0% -99.50% (p=0.000 n=10+10)
2020-10-07 20:47:27 +02:00
Michael Eischer
b8eacd1364
filter: Reduce redundant path and pattern splitting
...
A single call to filter.List will split the path only once and also
split each search pattern only once and use it for both match and
childMatch.
name old time/op new time/op delta
FilterPatterns/Relative-4 62.1ms ±15% 30.3ms ±10% -51.22% (p=0.000 n=9+10)
FilterPatterns/Absolute-4 111ms ±10% 49ms ± 3% -56.08% (p=0.000 n=10+8)
FilterPatterns/Wildcard-4 393ms ±15% 345ms ± 9% -12.30% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 10.0s ± 3% 3.9s ± 2% -60.53% (p=0.000 n=10+9)
name old alloc/op new alloc/op delta
FilterPatterns/Relative-4 16.4MB ± 0% 4.6MB ± 0% -71.76% (p=0.000 n=10+9)
FilterPatterns/Absolute-4 31.4MB ± 0% 8.5MB ± 0% -72.77% (p=0.000 n=9+10)
FilterPatterns/Wildcard-4 168MB ± 0% 146MB ± 0% -13.19% (p=0.000 n=10+9)
FilterPatterns/ManyNoMatch-4 3.23GB ± 0% 0.91GB ± 0% -71.96% (p=0.000 n=10+9)
name old allocs/op new allocs/op delta
FilterPatterns/Relative-4 178k ± 0% 67k ± 0% -62.50% (p=0.000 n=10+10)
FilterPatterns/Absolute-4 266k ± 0% 89k ± 0% -66.67% (p=0.000 n=10+10)
FilterPatterns/Wildcard-4 1.87M ± 0% 1.70M ± 0% -9.47% (p=0.000 n=10+10)
FilterPatterns/ManyNoMatch-4 17.7M ± 0% 4.5M ± 0% -74.87% (p=0.000 n=9+10)
2020-10-07 18:13:19 +02:00
Michael Eischer
e73c281142
filter: Benchmark absolute paths, wildcards and long filter lists
2020-10-07 17:54:36 +02:00
Michael Eischer
fdd3b14db3
filter: test some corner cases
2020-10-07 17:09:44 +02:00
Michael Eischer
f4282aa6fd
local: mark repository files as read-only
...
This is intended to prevent accidental modifications of data files.
Marking the files as read-only was accidentally removed in #1258 .
2020-10-07 12:29:37 +02:00
Michael Eischer
40ee17167e
local: Ignore permission errors on chmod call in Save/Remove operation
...
The file is already created with the proper permissions, thus the chmod
call is not critical. However, some file systems have don't allow
modifications of the file permissions. Similarly the chmod call in the Remove
operation should not prevent it from working.
2020-10-07 12:29:37 +02:00
greatroar
f3e933f0c1
Use windows.SetConsoleCursorPosition in ui/termstatus
2020-10-07 11:18:54 +02:00
MichaelEischer
0ae02f3030
Merge pull request #2844 from aawsome/prune-integration-tests
...
Add prune integration tests for many edge cases
2020-10-06 23:16:41 +02:00
rawtaz
e401859afb
Merge pull request #2992 from lbausch/repo_wording
...
Replace repo/repos with repository/repositories in `--no-lock` help text
2020-10-06 23:03:55 +02:00
Lorenz Bausch
af4100e07d
Replace repo/repos with repository/repositories in --no-lock help text
2020-10-06 22:49:48 +02:00
Alexander Neumann
ea5bbe0857
Remove dysfunctional link to saythanks
2020-10-06 20:43:44 +02:00
Alexander Weiss
6822a58413
Add prune integration tests for many edge cases
2020-10-06 20:20:05 +02:00
MichaelEischer
09d39e260d
Merge pull request #2980 from greatroar/error-wrapping
...
Remove repetitive error wrapping from internal/cache
2020-10-06 15:19:00 +02:00
Michael Eischer
1579d2a8ec
Remove some unused assignments
2020-10-06 14:55:13 +02:00
Michael Eischer
eba5dd831f
Fix typos reported by misspell
2020-10-06 14:55:13 +02:00
Michael Eischer
9ffb698c8d
Modernize context import in internal/fuse
...
Migrated using `go fix`.
2020-10-06 14:55:13 +02:00
Michael Eischer
efbb850d92
Remove a few redundant type specifiers
...
This is the result of running `gofmt -s -w **/*.go`
2020-10-06 14:55:13 +02:00
MichaelEischer
cfd57c480a
Merge pull request #2991 from restic/nonsense
...
cache/check: Clarify error message when given arguments
2020-10-06 00:40:12 +02:00
Leo R. Lundgren
e105a3f391
cache/check: Clarify error message when given arguments
2020-10-06 00:08:59 +02:00
rawtaz
5d8cfff3f2
Merge pull request #2988 from greatroar/snapshots-json
...
Regression test for #2979
2020-10-05 21:02:52 +02:00
greatroar
673dda77c0
Less repetitive error messages in internal/cache
...
Many instances of errors.Wrap in this package would produce messages
like "Open: open <filename>: no such file or directory"; those now omit
the first "Open:" (or "Stat:", or "MkdirAll"). The function readVersion
now appends its own name to the error message, rather than the function
that failed, to make it easier to spot. Other function names (e.g.,
Load) are already added further up in the call chain.
2020-10-05 20:28:54 +02:00
MichaelEischer
1ab4c710e1
Merge pull request #2987 from greatroar/no-password-for-help
...
Don't read password for generate, help or self-update
2020-10-05 20:16:19 +02:00
greatroar
feedf0ebce
Don't read password for generate, help or self-update
...
Fixes #2951 .
2020-10-05 19:54:07 +02:00
MichaelEischer
1a490acd67
Merge pull request #2982 from greatroar/archiver-error-handling
...
Check error in archiver before calling Select
2020-10-05 19:19:47 +02:00
Michael Eischer
187518a8a3
docs: Properly reproduce zip archives for windows binaries
2020-10-05 18:24:46 +02:00
Michael Eischer
a232c833dc
docs: Update binary reproduction instructions and refer to older instructions
2020-10-05 18:24:46 +02:00
MichaelEischer
c84643c6a9
Merge pull request #2981 from greatroar/mkdircachedir
...
Simplify cache directory creation
2020-10-05 18:15:23 +02:00
MichaelEischer
8390a8aaf3
Merge pull request #2986 from greatroar/upgrade-xxhash
...
Upgrade cespare/xxhash to 2.1.1
2020-10-05 17:49:08 +02:00
greatroar
1e1a1f3078
Upgrade cespare/xxhash to 2.1.1
2020-10-05 17:25:39 +02:00
MichaelEischer
abe9fa261f
Merge pull request #2983 from greatroar/upgrade-siphash
...
Upgrade siphash to 1.2.2
2020-10-05 17:18:07 +02:00
MichaelEischer
6001b45bf7
Merge pull request #2984 from restic/snapshots-json
...
snapshots: Make --json output [] instead of null when no snapshots
2020-10-05 17:14:23 +02:00
greatroar
6bee0aafc2
Regression test for #2979
2020-10-05 15:08:09 +02:00
greatroar
8252ea8e3d
Upgrade siphash to 1.2.2
...
New version is potentially faster on 32-bit ARM.
2020-10-05 14:56:13 +02:00
Leo R. Lundgren
137d20a06a
snapshots: Make --json output [] instead of null when no snapshots
2020-10-05 13:14:10 +02:00
greatroar
c4e2203e45
Check error in archiver before calling Select
...
The archiver first called the Select function for a path before checking
whether the Lstat on that path actually worked. The RejectFuncs in
exclude.go worked around this by checking whether they received a nil
os.FileInfo. Checking first is more obvious and requires less code.
2020-10-05 11:11:04 +02:00
greatroar
7d0fa1a686
Simplify cache directory creation
2020-10-05 10:46:45 +02:00
rawtaz
a23d90d270
Merge pull request #2975 from restic/rawtaz-doc-resume
...
Add FAQ on whether restic can resume backups
2020-10-03 23:12:34 +02:00
MichaelEischer
2630411530
Merge pull request #2974 from gottwald/swap-gcs-lib
...
Swap deprecated GCS lib with replacement
2020-10-03 21:58:11 +02:00
rawtaz
b9b82d878d
Add FAQ on whether restic can resume backups
...
This new FAQ entry explains that restic can resume interrupted backups.
2020-10-03 21:39:45 +02:00
Ingo Gottwald
8b8e230771
Swap deprecated GCS lib with replacement
2020-10-03 18:55:56 +02:00
rawtaz
7c3c6fa431
Merge pull request #2977 from MichaelEischer/downgrade-cobra
...
Downgrade cobra to v0.0.5
2020-10-03 18:23:13 +02:00
Michael Eischer
29908906b7
Downgrade cobra to v0.0.5
2020-10-03 18:13:07 +02:00
rawtaz
bbeb439f41
Merge pull request #2937 from andreaso/self-update-output-path-fix
...
Don't require `self-update --output` placeholder file
2020-10-03 13:35:20 +02:00
Andreas Olsson
ce14df303b
Don't require `self-update --output` placeholder file
...
This removes the requirement on `restic self-update --output` to point
to a path of an existing file, to overwrite. In case the specified
path does exist we still want to verify that it's a regular file,
rather than a directory or a device, which gets overwritten.
We also want to verify that a path to a new file exists within an
existing directory. The alternative being running into that issue
after the actual download, etc has completed.
While at it I also replace `errors.Errorf` with the more appropriately
verbose `errors.Fatalf`.
Resolves #2491
2020-10-03 07:02:23 +02:00
MichaelEischer
3c6671b18b
Merge pull request #2973 from restic/rawtaz-repo2
...
Correct data type for --repo2 option
2020-10-02 22:22:41 +02:00
MichaelEischer
86ff1f2bf6
Merge pull request #2972 from restic/rawtaz-verbose
...
Clarify max verbose level in option description
2020-10-02 22:21:45 +02:00
rawtaz
0cce6dc31c
Correct data type for --repo2 option
2020-10-02 21:10:52 +02:00
rawtaz
6253ff0187
Clarify max verbose level in option description
...
Clarifies that the max verbosity level is 3, in the description of the --verbose option.
2020-10-02 20:25:34 +02:00
rawtaz
e9943e864f
Merge pull request #2955 from jtagcat/clarify--compact
...
help: --compact: clarify description
2020-10-02 15:59:08 +02:00
jtagcat
a687261804
--help: --compact: clarify description
2020-10-02 16:55:56 +03:00
MichaelEischer
4df8861e09
Merge pull request #2967 from gottwald/fix-deprctd-gs-constructor
...
Replace deprecated method in gs backend
2020-10-01 11:01:49 +02:00
Ingo Gottwald
00cedd22aa
Replace deprecated method in gs backend
2020-10-01 10:02:42 +02:00
MichaelEischer
f361ed66de
Merge pull request #2910 from mtdcr/repository-file
...
Add new option --repository-file (default: $RESTIC_REPOSITORY_FILE)
2020-10-01 01:04:23 +02:00
Michael Eischer
7b50a65492
Update backup help output in documentation
2020-10-01 00:50:27 +02:00
Michael Eischer
c18b119a9b
Document new option --repository-file
2020-10-01 00:50:27 +02:00
Michael Eischer
61035d68bc
Add test for --repository-file
2020-10-01 00:50:27 +02:00
Andreas Oberritter
97f7855de3
Add new option --repository-file (default: $RESTIC_REPOSITORY_FILE)
...
As an alternative to -r, this allows to read the repository URL
from a file in order to prevent certain types of information leaks,
especially for URLs containing credentials.
Fixes #1458 , fixes #2900 .
2020-10-01 00:50:26 +02:00
Michael Eischer
d44df9d00d
backup: Always remove the status lines once a backup ends
2020-09-30 23:13:10 +02:00
Michael Eischer
8d0ba55ecd
backup: Fix possible deadlock of scanner goroutine
...
When the backup is interrupted for some reason while the scanner is
still active this could lead to a deadlock. Interruptions are triggered
by canceling the context object used by both the backup progress UI and
the scanner. It is possible that a context is canceled between the
respective check in the scanner and it calling the `ReportTotal` method
of the UI. The latter method sends a message to the UI goroutine.
However, a canceled context will also stop that goroutine, which can
cause the channel send operation to block indefinitely.
This is resolved by adding a `closed` channel which is closed once the
UI goroutine is stopped and serves as an escape hatch for reported UI
updates.
This change covers not just the ReportTotal method but all potentially
affected methods of the progress UI implementation.
2020-09-30 23:13:10 +02:00
rawtaz
34ea960559
Merge pull request #2966 from MichaelEischer/recover-help-typo
...
recover: Fix typo in the command help
2020-09-30 18:16:26 +02:00
rawtaz
5ea01b00df
Merge pull request #2965 from MichaelEischer/rework-password-prompt
...
Clarify verbose password prompt
2020-09-30 18:15:05 +02:00
MichaelEischer
48d0ab5276
Merge pull request #2959 from restic/keep-text
...
backup: Correct keep policy text
2020-09-30 17:57:00 +02:00
Michael Eischer
2a79c1a44d
recover: Fix typo in the command help
...
Reported-by: Steve Divskinsy <stevesbrain@users.noreply.github.com>
2020-09-30 17:44:34 +02:00
MichaelEischer
fd02407863
Merge pull request #2849 from classmarkets/gcs-access-token
...
gs: support authentication with access token
2020-09-30 17:42:56 +02:00
Michael Eischer
aea9f7d286
clarify verbose password prompt
2020-09-30 17:25:54 +02:00
Leo R. Lundgren
028c9a5343
backup: Correct keep policy text
...
Makes the following corrections to the "Applying Policy:" output:
- keep the last 1 snapshots snapshots => keep 1 latest snapshots
- keep the last 1 snapshots, 3 hourly, 5 yearly snapshots => keep 1 latest, 3 hourly, 5 yearly snapshots
2020-09-28 14:26:53 +02:00
MichaelEischer
94136132e3
Merge pull request #2957 from plumbeo/patch-3
...
Don’t print excessively detailed debug messages on object deletion when —verbose is used
2020-09-27 22:08:07 +02:00
plumbeo
009bd907f2
Don’t print too many messages when deleting files
...
Print detailed debug messages on file deletions only when --verbose=2 is used
2020-09-27 14:24:04 +02:00
MichaelEischer
14b312f00d
Merge pull request #2658 from creativeprojects/issue-2241
...
Don't echo authentication passwords (rest backend)
2020-09-22 22:17:53 +02:00
Fred
206cadfab4
Hide password from repository URLs
2020-09-22 22:00:51 +02:00
MichaelEischer
4875f7b659
Merge pull request #2614 from greatroar/simplify-fs
...
Simplify internal/fs
2020-09-21 22:05:10 +02:00
MichaelEischer
6f2093e491
Merge pull request #2940 from andreaso/appveyor-tar-https
...
Use https:// to download appveyor tar
2020-09-21 21:50:26 +02:00
MichaelEischer
16f31b2f73
Merge pull request #2939 from J0WI/patch-1
...
Update Go version to 1.15 in Docker build script
2020-09-21 21:46:33 +02:00
greatroar
0d65b78168
Simplify os.ModeType|os.ModeCharDevice => os.ModeType
...
Since Go 1.12, ModeCharDevice is included in ModeType:
golang/go@a2a3dd00c9
2020-09-21 14:21:32 +02:00
greatroar
95ebba85ff
Remove stray Printf from internal/fs
2020-09-21 14:21:32 +02:00
greatroar
59b343a9bf
Remove OS-specific versions of fs.MkdirAll
...
Go has supported Windows paths correctly since 1.11, see
https://github.com/golang/go/issues/10900 and the commit referenced
there.
2020-09-21 14:21:32 +02:00
greatroar
1557c58eef
Fix and simplify fs.Reader
...
fakeDir.{Readdir,Readdirnames} weren't handling the case n == 0
correctly. fakeFileInfo.sys is always nil, so omit the field.
2020-09-21 14:21:32 +02:00
greatroar
c504aa505c
Remove unused fs.Rename
2020-09-21 14:21:32 +02:00
greatroar
1b20f6beec
Remove io.Writer from fs.File
...
It was only used in a single test, which now uses plain *os.File instead.
2020-09-21 14:21:32 +02:00
rawtaz
10e3340863
Merge pull request #2945 from YoshieraHuang/max-file-size
...
Fix nil check in rejectBySize
2020-09-21 14:18:45 +02:00
yoshiera
3cf29a777d
Fix nil check in rejectBySize
2020-09-21 19:20:24 +08:00
Andreas Olsson
fd1f7b7268
Use https:// to download appveyor tar
...
When supported one might as well use https://.
2020-09-20 18:49:46 +02:00
J0WI
090a73f7c5
Update Go version to 1.15 in Docker build script
2020-09-20 15:42:12 +00:00
rawtaz
1cfb01a8a6
Merge pull request #2936 from andreaso/contrib-update-version
...
Update versions in contribution documentation
2020-09-20 13:27:13 +02:00
Andreas Olsson
ce62d3d689
Update versions in contribution documentation
...
Reflects 3c44598
and 429f97b
.
2020-09-20 10:32:27 +02:00
Michael Eischer
8c36317b71
rclone: use configured number of connections during create
2020-09-19 19:11:43 +02:00
Michael Eischer
60a5c35de9
rclone: close connection to rclone if open fails
2020-09-19 19:11:43 +02:00
Michael Eischer
9333f707fa
init: use Create method for rclone backend
...
This properly issues the initial repository creation command
Fixes #1896
2020-09-19 19:11:43 +02:00
Alexander Neumann
429f97b887
Set development version for 0.10.0
2020-09-19 17:38:47 +02:00
Alexander Neumann
40832b2927
Add version for 0.10.0
2020-09-19 17:38:26 +02:00
Alexander Neumann
c8a94eced7
Update manpages and auto-completion
2020-09-19 17:38:26 +02:00
Alexander Neumann
ee6e981b4e
Generate CHANGELOG.md for 0.10.0
2020-09-19 17:38:10 +02:00
Alexander Neumann
96fd982f6a
Prepare changelog for 0.10.0
2020-09-19 17:37:59 +02:00
Alexander Neumann
6ff0082c02
Merge pull request #2928 from MichaelEischer/init-copy-chunker
...
init: Add `--copy-chunker-parameters` option
2020-09-19 17:32:13 +02:00
Alexander Neumann
95c1d7d959
Merge pull request #2927 from restic/update-deps
...
Update dependencies
2020-09-19 17:29:35 +02:00
Michael Eischer
07f4e7d10b
Only log HTTP requests when a debug log is configured
...
The logged HTTP requests are only visible when a debug log is
configured.
2020-09-19 17:07:53 +02:00
Michael Eischer
f003410402
init: Add `--copy-chunker-params` option
...
This allows creating multiple repositories with identical chunker
parameters which is required for working deduplication when copying
snapshots between different repositories.
2020-09-19 16:53:05 +02:00
Michael Eischer
9ad8250a78
Reduce overhead of debug calls if no log is enabled
...
In case no debug log is configured, then calls to debug.Log only incur
the costs to check a single boolean flag making the call really cheap.
2020-09-19 16:45:01 +02:00
Michael Eischer
655430550b
Extract parameters for second repository from copy command
2020-09-19 16:07:55 +02:00
MichaelEischer
1823b8195c
Merge pull request #2930 from rawtaz/password-file-text
...
Improve wording for --password-file and related options
2020-09-19 16:06:15 +02:00
Leo R. Lundgren
311ad2d2d0
Improve wording for --password-file and related options
2020-09-19 15:47:32 +02:00
Alexander Neumann
a10b44a265
Add note about fuse
2020-09-19 14:35:55 +02:00
Alexander Neumann
baf3a9aa3b
Merge pull request #2929 from rawtaz/password-command-text
...
Improve wording for --password-command and --password-command2 options
2020-09-19 14:30:39 +02:00
Leo R. Lundgren
ffe6dce7e7
Improve wording for --password-command and --password-command2 options
2020-09-19 14:23:25 +02:00
Alexander Neumann
8ce0ce387f
Add changelog
2020-09-19 14:22:31 +02:00
Alexander Neumann
3c44598bf6
Drop Go 1.12, require at least 1.13
2020-09-19 14:20:04 +02:00
Alexander Neumann
3bb55fd6bf
Merge pull request #2914 from YoshieraHuang/max-file-size
...
Support excluding files by size
2020-09-19 14:12:15 +02:00
Alexander Neumann
36efefa7bd
Update dependencies, pin fuse library
...
This updates all dependencies which causes the following changes:
* The fuse library is set to the last version supporting macOS
* The minimal version of Go required to build restic is now 1.13
2020-09-19 14:06:37 +02:00
yoshiera
ac4b8c98ac
Support excluding files by size
2020-09-19 19:52:09 +08:00
MichaelEischer
4dcd6abf37
Merge pull request #2616 from greatroar/no-cache-interface
...
Remove practically unused cache interface
2020-09-18 19:50:47 +02:00
greatroar
cb3f531050
Make Cache.Path private
...
It's only used inside the package.
2020-09-18 11:17:29 +02:00
greatroar
23fcbb275a
Remove restic.Cache interface
...
It was used in one code path, which then asserted its concrete type as
*cache.Cache.
Privatised some of the interface methods.
2020-09-18 10:48:13 +02:00
greatroar
6e3215a80d
Remove unused error type from internal/cache
2020-09-18 10:38:14 +02:00
greatroar
9abef3bf1a
Move internal/fs.TestChdir to internal/test.Chdir
2020-09-17 10:43:33 +02:00
Alexander Neumann
b10dce541e
Merge pull request #2598 from MichaelEischer/diff-speedup
...
Diff speedup
2020-09-13 16:42:05 +02:00
Michael Eischer
4f221c4022
Add changelog entry
2020-09-13 16:28:46 +02:00
Michael Eischer
f5c448aa65
diff: Optimize diff calculation for shared subtrees
...
When the diff calculation compares two trees with identical id then no
differences between them can ever show up. Optimize for that case by
simply traversing the tree only once to collect all referenced blobs for
a proper calculation of added and removed blobs.
Just skipping the common subtrees is not possible as this would skew the
results if the added or removed blobs are shared with one of the
subtrees.
2020-09-13 16:28:46 +02:00
Michael Eischer
c0fc85d303
diff: Add integration test
2020-09-13 16:28:46 +02:00
Alexander Neumann
0c48e515f0
Merge pull request #2630 from MichaelEischer/fix-staticcheck
...
Fix lots of small issues reported by staticcheck
2020-09-13 16:19:22 +02:00
Alexander Neumann
97950ab81a
options: Fix test for Go >= 1.15
2020-09-12 17:36:44 +02:00
Alexander Neumann
59fca85844
Remove darwin/386 from integration tests
2020-09-12 17:09:33 +02:00
Alexander Neumann
e207257714
Fix Appveyor
2020-09-12 16:59:03 +02:00
Alexander Neumann
82e1cbed4f
Update Go versions
2020-09-12 16:52:21 +02:00
Alexander Neumann
8903b6c88a
helpers: Rework list of architectures
...
This also removes darwin/386 which is not supported on Go 1.15 any more.
2020-09-12 16:39:41 +02:00
Alexander Neumann
93583c01b1
helpers: Fetch modules for release binaries
2020-09-12 16:25:36 +02:00
MichaelEischer
88664ba222
Merge pull request #2893 from MichaelEischer/restore-preallocate
...
restorer: Preallocate files
2020-09-08 22:43:05 +02:00
Michael Eischer
121233e1b3
Add preallocate tests
2020-09-07 21:41:47 +02:00
Michael Eischer
8cc9514879
restorer: pre-allocate files before loading chunks
2020-09-07 21:41:47 +02:00
Michael Eischer
2e7d475029
Process packs in order of first appearance
2020-09-06 21:25:35 +02:00
rawtaz
d3a286928a
Merge pull request #2905 from MichaelEischer/fix-ls-help
...
ls: Explicitly enforce that the user specifies a snapshot ID
2020-09-05 10:44:01 +02:00
Michael Eischer
c46edcd9d6
error strings should not end with punctuation
2020-09-05 10:07:17 +02:00
Michael Eischer
1ede018ea6
error variable names should start with 'Err'
2020-09-05 10:07:17 +02:00
Michael Eischer
b77e933d80
Convert underscore variable names to camelCase
2020-09-05 10:07:16 +02:00
Michael Eischer
d0329cf3eb
Adjust comments to match name of exported methods
2020-09-05 10:07:16 +02:00
Michael Eischer
dc31529fc3
Unindent else block after if block ending with a return statement
2020-09-05 10:07:16 +02:00
Michael Eischer
4784540f04
repository: Simplify worker group code
2020-09-05 10:07:16 +02:00
Michael Eischer
84ea2389ae
archiver_test: Only skip symlinks on windows
2020-09-05 10:07:16 +02:00
Michael Eischer
b4a7ce86cf
uint cannot be less than zero
2020-09-05 10:07:16 +02:00
Michael Eischer
7ee0964880
crypto: Remove unused error
2020-09-05 10:07:16 +02:00
Michael Eischer
f6f11400c2
restic: Remove unused variable value
2020-09-05 10:07:16 +02:00
Michael Eischer
e2dc5034d3
restic/file: Use FileType for all File constants
...
The data type of a variable is only carried on when using iota.
2020-09-05 10:07:16 +02:00
Michael Eischer
9a1b3cb5d9
restorer: Remove unused treeID struct member from test
2020-09-05 10:07:16 +02:00
Michael Eischer
b22655367c
integration_test: Replace fprintf without format string
2020-09-05 10:07:16 +02:00
Michael Eischer
068a3ce23f
Remove redundant return
2020-09-05 10:07:16 +02:00
Michael Eischer
ee05501ce7
archiver: Remove unused chmod function
2020-09-05 10:07:16 +02:00
Michael Eischer
014600bee6
archiver: Remove unused variable from test
2020-09-05 10:07:16 +02:00
Michael Eischer
d9a80e07b9
repository: Simplify index age calculation
2020-09-05 10:07:16 +02:00
Michael Eischer
d19f05c960
Use buf.String() instead of string(buf.Bytes())
2020-09-05 10:07:16 +02:00
Michael Eischer
460e2ffbf6
Collapse a few boolean operations
2020-09-05 10:07:14 +02:00
Michael Eischer
49b6aac3fa
Use !bytes.Equal instead of bytes.Compare !=0 to check for inequality
2020-09-05 10:06:23 +02:00
Michael Eischer
2f8335554c
Remove a few unused variables
2020-09-05 10:06:23 +02:00
Michael Eischer
37113282ca
Merge a few variable declaration and initializations
2020-09-05 10:05:34 +02:00
Michael Eischer
337725c354
Use sort.Strings
2020-09-05 10:05:34 +02:00
MichaelEischer
2ddb7ffb7e
Merge pull request #2255 from Kidswiss/tar
...
Fix dumping issues with / and the first sub level
2020-09-01 21:52:17 +02:00
Michael Eischer
81dcfea11a
dump: Mention tar output in command help text
2020-08-31 22:43:10 +02:00
MichaelEischer
55071ee367
Merge pull request #2859 from buschjost/stats-filter-by-tag-and-path
...
Add filter by tag and path to stats command
2020-08-31 22:11:01 +02:00
Michael Eischer
dcf9ded977
dump: Use slashes as path separators in tar on windows
...
This conversion is necessary as paths returned by filepath.Rel use the
operating-system native path separator
2020-08-30 18:26:50 +02:00
Michael Eischer
bcf44a9c3f
dump: Properly handle file paths without / prefix
...
filepath.Rel failed if the requested path did not start with a `/` e.g.
`restic` instead of `/restic`.
2020-08-30 18:25:42 +02:00
Oliver Buschjost
a7b4c19abf
Fix display of warning about ignored filters
2020-08-30 17:14:56 +02:00
Oliver Buschjost
d3fcfeba3a
Add filter by tag and path to stats command
2020-08-30 17:14:48 +02:00
Michael Eischer
e69449bf2c
dump: Add test for splitPath
2020-08-30 16:00:48 +02:00
Simon Beck
da4193c3ef
Add changelog
2020-08-30 16:00:48 +02:00
Simon Beck
fe6445e0f4
dump: Added unit tests for tar output
2020-08-30 16:00:48 +02:00
MichaelEischer
ea81a0e282
Merge pull request #2606 from middelink/fix-323
...
Add copy functionality.
2020-08-30 10:18:24 +02:00
Simon Beck
80a11960dd
dump: Always dump relative paths into tarballs
...
Tarballs should only contain relative paths.
2020-08-29 21:14:34 +02:00
Simon Beck
d6f739ec22
dump: Extract tar code to internal/dump
2020-08-29 21:14:34 +02:00
Simon Beck
b98598e55f
dump: Fix dumping issues with / and the first sub level
...
There was an issue that prevented the dump command from working
correctly when either:
* `/` contained multiple nodes (e.g. `restic backup /`)
* dumping a file in the first sublevel was attempted (e.g. `/foo`)
2020-08-29 21:14:29 +02:00
Michael Eischer
d5f86effa1
ls: Explicitly enforce that the user specifies a snapshot ID
...
The help messages suggested that the `ls` command work without
explicitly passing a snapshot ID. However, this was never the case:
without a snapshot ID the command just failed with the error
`Ignoring "", it is not a snapshot id`.
Fixes #2299
2020-08-29 14:28:53 +02:00
MichaelEischer
c34c731698
Merge pull request #2702 from MichaelEischer/fix-backup-dir-count
...
backup: Fix reporting of directory count in summary
2020-08-29 11:29:21 +02:00
Michael Eischer
412623b848
copy: Reuse buffer for downloaded blobs
2020-08-29 10:48:44 +02:00
Michael Eischer
bbe8b73f03
Update help text of backup command in docs
2020-08-29 10:48:44 +02:00
Michael Eischer
91e8d998cd
Add documentation for copy command
2020-08-29 10:48:44 +02:00
Michael Eischer
9a4796594a
integration tests: Fix checking of wrong snapshot
2020-08-29 10:48:44 +02:00
Michael Eischer
15374d22e9
integration tests: Add basic tests for copy command
2020-08-29 10:48:44 +02:00
Michael Eischer
88ad58d6cd
integration tests: Redirect directory diff into intermediate buffer
2020-08-29 10:48:44 +02:00
Michael Eischer
591a8c4cdf
integration tests: Deduplicate backup test-data setup code
2020-08-29 10:48:44 +02:00
Michael Eischer
ec9a53b7e8
copy: Mark and skip previously copied snapshots
...
Use the `Original` field of the copied snapshot to store a persistent
snapshot ID. This can either be the ID of the source snapshot if
`Original` was not yet set or the previous value stored in the
`Original` field. In order to still copy snapshots modified using the
tags command the source snapshot is compared to all snapshots in the
destination repository which have the same persistent ID. Snapshots are
only considered equal if all fields except `Original` and `Parent`
match. That way modified snapshots are still copied while avoiding
duplicate copies at the same time.
2020-08-29 10:48:44 +02:00
MichaelEischer
34a3adfd8d
Merge pull request #2898 from aawsome/add-iexclude-file
...
backup: Add --iexclude-file option
2020-08-28 21:59:16 +02:00
Alexander Weiss
9867c4bbb4
Add flag --iexclude-file to backup
2020-08-28 21:04:41 +02:00
MichaelEischer
efb4a981cf
Merge pull request #2904 from jasonccox/consistent-usage-strings
...
Update usage strings to put flags before args
2020-08-28 20:39:14 +02:00
Jason Cox
2447f3f110
Update usage strings to put flags before args
...
The standard UNIX-style ordering of command-line arguments places
optional flags before other positional arguments. All of restic's
commands support this ordering, but some of the usage strings showed the
flags after the positional arguments (which restic also parses just
fine). This change updates the doc strings to reflect the standard
ordering.
Because the `restic help` command comes directly from Cobra, there does
not appear to be a way to update the argument ordering in its usage
string, so it maintains the non-standard ordering (positional arguments
before optional flags).
2020-08-28 02:16:22 +00:00
Michael Eischer
b25978a53c
backup: Fix reporting of directory count in summary
...
Previously the directory stats were reported immediately after calling
`SaveDir`. However, as the latter method saves the tree asynchronously
the stats were still initialized to their nil value. The stats are now
reported via a callback similar to the one used for the fileSaver.
2020-08-27 22:43:51 +02:00
Michael Eischer
b0a8c4ad6c
copy: Only process each tree once
...
This speeds up copying multiple overlapping snapshots from one
repository to another, as we only have to copy the changed parts of
later snapshots.
2020-08-26 22:17:52 +02:00
Michael Eischer
908b23fda0
copy: Update for modernized repository interface
2020-08-26 22:17:52 +02:00
greatroar
4508d406ef
copy: Remove separate SaveIndex in restic copy
...
Flush does this now.
2020-08-26 22:17:52 +02:00
Pauline Middelink
7048cc3e58
Add copy functionality
...
Add a copy command to copy snapshots between repositories. It allows the user
to specify a destination repository, password, password-file, password-command
or key-hint to supply the necessary details to open the destination repository.
You need to supply a list of snapshots to copy, snapshots which already exist
in the destination repository will be skipped.
Note, when using the network this becomes rather slow, as it needs to read the
blocks, decrypt them using the source key, then encrypt them again using the
destination key before finally writing them out to the destination repository.
2020-08-26 22:17:52 +02:00
MichaelEischer
eb7c00387c
Merge pull request #2899 from MichaelEischer/fix-check-progress-crash
...
Unify progress bar of check and prune commands
2020-08-26 00:00:50 +02:00
Michael Eischer
bc0501d72c
Add changelog
2020-08-25 23:33:10 +02:00
Michael Eischer
17995dec7a
Unify progress bar of check and prune commands
2020-08-25 22:47:38 +02:00
MichaelEischer
e915cedc3d
Merge pull request #2897 from restic/rawtaz-github-issue-feature
...
github: Add question to feature issue template
2020-08-24 23:52:38 +02:00
MichaelEischer
cdcaecd27d
Merge pull request #2896 from restic/rawtaz-key-hint
...
doc: Add missing environment variables
2020-08-24 23:51:05 +02:00
rawtaz
b43ab67a22
doc: Add missing environment variables
2020-08-24 00:49:34 +02:00
rawtaz
7ddfd6cabe
github: Add question to feature issue template
2020-08-24 00:22:44 +02:00
Alexander Neumann
b1b3f1ecb6
Merge pull request #2674 from MichaelEischer/prune-strict-checks
...
prune: Stricter error checks
2020-08-23 10:30:41 +02:00
rawtaz
fa135f72bf
Merge pull request #2439 from htrendev/ignore-sync-on-macos
...
Ignore not supported error on sync() when using local backend
2020-08-22 01:38:49 +02:00
Hristo Trendev
51c22f4223
local backend: ignore not supported error on sync()
...
Closes #2395
2020-08-22 01:27:07 +02:00
rawtaz
1a5b66f33b
Merge pull request #2879 from 0xMH/master
...
Added more explantation for --one-file-system option
2020-08-21 19:21:01 +02:00
NoNE
da6a34e044
doc: Add more explanation for --one-file-system option
2020-08-21 19:19:13 +02:00
Bruce Dillahunty
fe69b83074
Update key password prompt ( #2847 )
2020-08-19 21:42:08 +02:00
Michael Eischer
08d24ff99e
prune: Include ID of all missing blobs in error message
2020-08-16 11:36:14 +02:00
Michael Eischer
d8b80e9862
Add changelog
2020-08-16 11:36:14 +02:00
Michael Eischer
1c84aceb39
prune: Test for abort on damaged repositories
2020-08-16 11:36:12 +02:00
Michael Eischer
575ed9a47e
Test that rebuild-index errors when old index cannot be removed
2020-08-16 11:34:01 +02:00
Michael Eischer
8f811642c3
Add support for integration tests to wrap the backend
2020-08-16 11:34:01 +02:00
Michael Eischer
f4b9544ab2
prune: Add test that repack aborts on wrong blob
2020-08-16 11:34:01 +02:00
Michael Eischer
367449dede
prune: Reduce memory allocations while repacking
...
The slicing operator `slice[low:high]` default to 0 for the lower bound and
len(slice) for the upper bound when either or both are not specified.
Fix the code to use `cap(slice)` to check for the slice capacity.
2020-08-16 11:34:01 +02:00
Michael Eischer
7042bafea5
prune: Abort repacking when a pack contains a wrong blob
...
If a blob in a pack file can be decrypted successfully but contains data
that results in a different hash than stated in the header pack, then
abort repacking. As both the pack header and the blob are
cryptographically verified this either means than a malicious entity
tampered with the backup or indicates hardware problems on the client.
prune should fail with an error in both cases.
2020-08-16 11:34:01 +02:00
Michael Eischer
744a15247d
prune/rebuild-index: Fail if an old index cannot be removed
...
The old behavior was problematic in the context of rebuild-index as it
could leave old, possibly invalid index files behind without returning a
fatal error.
Prune calls rebuildIndex before removing any data from the repository.
For this use case failing to delete an old index MUST be treated as a
fatal error. Otherwise the index could still contain an old index file
that refers to blobs/packs that were later on deleted by prune. Later
backup runs will assume that the affected blobs already exist in the
repository which results in a backup which misses data.
2020-08-16 11:34:01 +02:00
Michael Eischer
3ba19869be
prune: Abort if any used blobs are missing
...
The previous check only approximately verified whether all required
blobs were found. However, after forgetting a few snapshots the
repository contains lots of unused blobs whose number can be sufficient
to make up for missing packs.
When coupled with a malfunctioning backend that temporarily returns broken
data this could cause restic to regard the corresponding packs as
invalid and thereby delete data that's still in use. This change lets
restic play it safe and refuse to delete anything if data is missing.
2020-08-16 11:34:01 +02:00
aawsome
0fed6a8dfc
Use "pack file" instead of "data file" ( #2885 )
...
- changed variable names, especially changed DataFile into PackFile
- changed in some comments
- always use "pack file" in docu
2020-08-16 11:16:38 +02:00
rawtaz
643bbbe156
doc: Correct wording for repository ( #2884 )
...
Replaces "backend" with "repository" to match restic output.
2020-08-11 22:30:17 +02:00
MichaelEischer
eca0f0ad24
Merge pull request #2863 from aawsome/index-no-duplicates
...
Don't save exact duplicates in merged index
2020-08-08 18:24:14 +02:00
MichaelEischer
08dee8a52b
Merge pull request #2865 from greatroar/unused
...
Dead code removal
2020-08-08 16:15:28 +02:00
Alexander Weiss
b112533812
Don't save exact duplicates when merging indexes
2020-08-05 06:32:02 +02:00
Alexander Weiss
5e63294355
Add benchmark MasterIndexAlloc
2020-08-05 06:32:02 +02:00
MichaelEischer
84b6f1ec53
Merge pull request #2874 from MichaelEischer/fix-verbose
...
Adjust description of --verbose=n parameter
2020-08-04 23:21:05 +02:00
MichaelEischer
06fb4ea3f0
Merge pull request #2805 from jwilk-forks/verbose
...
Fix examples of --verbose with argument
2020-08-04 23:11:35 +02:00
Michael Eischer
e38d415173
Adjust description of --verbose=n parameter
2020-08-04 23:07:53 +02:00
greatroar
d81a396944
Dead code removal
...
Found by running golangci-lint on the entire code base.
2020-08-04 08:38:57 +02:00
rawtaz
0b21ec44b7
Merge pull request #2869 from josephrocca/patch-1
...
doc: --verbose 2 to --verbose=2
2020-08-04 01:06:58 +02:00
josephrocca
bd36731119
`--verbose 2` to `--verbose=2`
...
`--verbose 2` seems to be incorrect here (gives an error/warning that "the `2` directory does not exist, skipping")
2020-08-04 08:39:43 +10:00
MichaelEischer
38a2f9c07b
Merge pull request #2864 from greatroar/dump-error
...
Fix error handling in dump (err != err)
2020-08-03 20:09:42 +02:00
rawtaz
5af2815627
Merge pull request #2821 from renard/mount-nolock
...
cmd/mount: honur --no-lock flag
2020-08-03 20:09:26 +02:00
MichaelEischer
b55de2260d
Merge pull request #2868 from MichaelEischer/fix-getusedblobs
...
prune: Stop progress bar after searching used blobs
2020-08-03 19:58:59 +02:00
Sébastien Gross
9be4fe3e84
cmd/mount: honour --no-lock flag
...
Do not lock the repository if --no-lock global flag is set. This allows
to mount repositories which are archived on a read only system.
Signed-off-by: Sébastien Gross <seb•ɑƬ•chezwam•ɖɵʈ•org>
2020-08-03 19:45:39 +02:00
Michael Eischer
05116e4787
prune: Cleanup progress bar handling while repacking
2020-08-03 19:32:46 +02:00
Michael Eischer
04f79b9642
prune: Stop progress bar after searching used blobs
2020-08-03 19:31:49 +02:00
greatroar
8b358935a0
Fix error handling in dump (err != err)
2020-08-03 09:43:44 +02:00
MichaelEischer
66d089e239
Merge pull request #2841 from aawsome/optimize-getUsedBlobs
...
Extract get used blobs in prune as separate function
2020-08-01 22:40:33 +02:00
MichaelEischer
49d3efe547
Merge pull request #2840 from aawsome/delete-parallel
...
Make delete parallel
2020-08-01 22:26:15 +02:00
Alexander Weiss
9762bec091
Use optimized getUsedBlobs in prune
2020-08-01 21:07:31 +02:00
Alexander Weiss
0eb8553c87
add changelog for #2840
2020-08-01 20:43:18 +02:00
Alexander Weiss
d3692f5b81
Delete files in parallel in rebuild-index
2020-08-01 20:43:18 +02:00
Alexander Weiss
1c0b61204b
Delete files in parallel in forget
2020-08-01 20:43:18 +02:00
Alexander Weiss
2ee654763b
Delete files in parallel
2020-08-01 20:39:24 +02:00
Alexander Neumann
b7b479b668
Merge pull request #2599 from MichaelEischer/tweak-mem-usage
...
Reduce memory usage when searching for used blobs
2020-08-01 13:49:02 +02:00
Alexander Neumann
4cf9656f12
Merge pull request #2861 from MichaelEischer/fix-rclone-crash
...
rclone: Don't panic after unexpected subprocess exit
2020-08-01 13:08:50 +02:00
Alexander Neumann
2580eef2aa
Merge pull request #2318 from classmarkets/2175-named-keys
...
Allow specifying user and host when adding keys
2020-08-01 13:06:31 +02:00
Michael Eischer
2d7ab9115f
Add changelog entry
2020-08-01 12:29:16 +02:00
Michael Eischer
a178e5628e
Remove duplicate TreeLoader interface
2020-08-01 12:29:16 +02:00
Michael Eischer
af66a62c04
FindUsedBlobs: Test that seen blobs are skipped
...
This also copies the TreeLoader interface from internal/walker to allow
stubbing the repository in the call to `FindUsedBlobs`.
2020-08-01 12:29:16 +02:00
Michael Eischer
9ea1a78bd4
FindUsedBlobs: Check for seen blobs before loading trees
...
The only effective change in behavior is that that toplevel nodes can
also be skipped.
2020-08-01 12:29:16 +02:00
Michael Eischer
184103647a
FindUsedBlobs: merge seen into blobs BlobSet
...
The seen BlobSet always contained a subset of the entries in blobs.
Thus use blobs instead and avoid the memory overhead of the second set.
Suggested-by: Alexander Weiss <alex@weissfam.de>
2020-08-01 12:29:16 +02:00
Michael Eischer
c81b122374
rclone: Don't panic after unexpected subprocess exit
...
As the connection to the rclone child process is now closed after an
unexpected subprocess exit, later requests will cause the http2
transport to try to reestablish a new connection. As previously this never
should have happened, the connection called panic in that case. This
panic is now replaced with a simple error message, as it no longer
indicates an internal problem.
2020-08-01 12:17:40 +02:00
rawtaz
48f97f3567
Merge pull request #2857 from MichaelEischer/incomplete-backup-error
...
Don't print a stacktrace if some files could not be read
2020-07-29 00:06:25 +02:00
Michael Eischer
3ce9893e0b
Don't print a stacktrace if some files could not be read
2020-07-28 23:52:48 +02:00
MichaelEischer
248c7c3828
Merge pull request #2587 from aawsome/optimize-fuse
...
Make `restic mount` faster and consume less memory
2020-07-28 23:19:17 +02:00
Alexander Weiss
f8316948d1
Optimize FUSE - make command `restic mount` faster and consume less memory
...
- Add Open() functionality to dir
- only access index for blobs when file is read
- Implement NodeOpener and put one-time file stuff there
- Add comment about locking as suggested by bazil.org/fuse
=> Thanks at Michael Eischer for suggesting the last two improvements
2020-07-28 23:01:18 +02:00
MichaelEischer
be54ceff66
Merge pull request #2855 from MichaelEischer/rclone-exit
...
Fix rclone subprocess handling
2020-07-26 22:55:25 +02:00
Michael Eischer
ea97ff1ba4
rclone: Skip crash test when rclone is not found
2020-07-26 12:06:18 +02:00
Michael Eischer
01b9581453
rclone: Better field names for stdio conn
2020-07-26 00:29:25 +02:00
Michael Eischer
3cd927d180
rclone: Give rclone time to finish before closing stdin pipe
...
Calling `Close()` on the rclone backend sometimes failed during test
execution with 'signal: Broken pipe'. The stdio connection closed both
the stdin and stdout file descriptors at the same moment, therefore
giving rclone no chance to properly send any final http2 data frames.
Now the stdin connection to rclone is closed first and will only be
forcefully closed after a timeout. In case rclone exits before the
timeout then the stdio connection will be closed normally.
2020-07-26 00:29:25 +02:00
greatroar
bf7b1f12ea
rclone: Add test for pipe handling when rclone exits
2020-07-26 00:29:25 +02:00
Michael Eischer
8554332894
rclone: Close rclone side of stdio_conn pipes
...
restic did not notice when the rclone subprocess exited unexpectedly.
restic manually created pipes for stdin and stdout and used these for the
connection to the rclone subprocess. The process creating a pipe gets
file descriptors for the sender and receiver side of a pipe and passes
them on to the subprocess. The expected behavior would be that reads or
writes in the parent process fail / return once the child process dies
as a pipe would now just have a reader or writer but not both.
However, this never happened as restic kept the reader and writer
file descriptors of the pipes. `cmd.StdinPipe` and `cmd.StdoutPipe`
close the subprocess side of pipes once the child process was started
and close the parent process side of pipes once wait has finished. We
can't use these functions as we need access to the raw `os.File` so just
replicate that behavior.
2020-07-26 00:29:25 +02:00
greatroar
3e93b36ca4
Make rclone.New private
2020-07-26 00:28:45 +02:00
MichaelEischer
573a2fb240
Merge pull request #2789 from aawsome/fix-lookup
...
make Lookup() return all blobs
2020-07-25 21:32:23 +02:00
Michael Eischer
c847aace35
Rename Index interface to MasterIndex
...
The interface is now only implemented by repository.MasterIndex.
2020-07-25 21:19:46 +02:00
Alexander Weiss
9d1fb94c6c
make Lookup() return all blobs
...
+ simplify syntax
2020-07-25 21:18:34 +02:00
MichaelEischer
020cab8e08
Merge pull request #2787 from greatroar/no-blobsize-cache
...
Remove blob size cache from restic mount
2020-07-25 20:46:35 +02:00
greatroar
07da61baee
Remove blob size cache from restic mount
2020-07-25 19:39:14 +02:00
MichaelEischer
37c95bf5da
Merge pull request #2835 from J0WI/patch-1
...
Update Go version to 1.14.6 in Docker build script
2020-07-25 17:23:54 +02:00
rawtaz
c86d2f23aa
Merge pull request #2854 from restic/rawtaz-patch-1
...
Clarify questions and text in PR template
2020-07-25 15:41:27 +02:00
rawtaz
96ec04d74d
Clarify questions and text in PR template
2020-07-25 15:38:00 +02:00
J0WI
9c3414374a
Update Go version to 1.14.6 in Docker build script
2020-07-25 14:45:36 +02:00
MichaelEischer
3d530dfc91
Merge pull request #2827 from aawsome/archiver-test-contents
...
Make self-healing work when backing up with parent snapshot
2020-07-25 13:13:18 +02:00
MichaelEischer
c43f5b2664
Merge pull request #2733 from hibbert/support_irsa
...
Enable support for EKS IRSA
2020-07-25 12:58:43 +02:00
Andy Hibbert
38087e40d9
Update minio-go version so it supports EKS IRSA
2020-07-25 12:41:18 +02:00
MichaelEischer
bbc960f957
Merge pull request #2635 from greatroar/optimize-sortbycached
...
Optimize sorting blobs by cache status
2020-07-25 12:35:42 +02:00
greatroar
309598c237
Simplify sortCachedPacksFirst test in internal/repository
...
The test now uses the fact that the sort is stable. It's not guaranteed
to be, but the test is cleaner and more exhaustive. sortCachedPacksFirst
no longer needs a return value.
2020-07-25 12:12:59 +02:00
greatroar
03d23e6faa
Speed up blob sorting in internal/repository
...
name old time/op new time/op delta
SortCachedPacksFirst-8 208µs ± 3% 186µs ± 3% -10.74% (p=0.000 n=10+8)
name old alloc/op new alloc/op delta
SortCachedPacksFirst-8 213kB ± 0% 139kB ± 0% -34.62% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
SortCachedPacksFirst-8 1.03k ± 0% 1.03k ± 0% -0.19% (p=0.000 n=10+10)
2020-07-25 12:12:59 +02:00
greatroar
b10acd2af7
Test and benchmark blob sorting in internal/repository
2020-07-25 12:12:58 +02:00
Alexander Weiss
9175795fdb
Check contents in archiver
...
When backing up with a parent snapshot and the file is not changed, also
check if contents are still available in index.
2020-07-25 08:18:28 +02:00
rawtaz
5d8d70542f
Merge pull request #2852 from MichaelEischer/drop-go-1.11
...
Drop support for Go version 1.11
2020-07-25 01:17:36 +02:00
Michael Eischer
7c23381a2b
Drop support for Go version 1.11
2020-07-24 18:52:39 +02:00
MichaelEischer
34181b13a2
Merge pull request #2328 from MichaelEischer/no-repeated-checks
...
Fix duplicate tree checks within `restic check`
2020-07-22 22:08:02 +02:00
MichaelEischer
bcd47ec3a2
Merge pull request #2818 from aawsome/merge-index-chaining
...
Merge index (based on chaining index implementation)
2020-07-22 22:05:36 +02:00
Alexander Weiss
a666a6d576
Add tests and merge indexes in index benchmarks
2020-07-22 21:54:02 +02:00
Alexander Weiss
e388d962a5
Merge final indexes together for faster index access
2020-07-22 21:54:02 +02:00
Peter Schultz
758b44b9c0
gs: support authentication with access token
...
In the Google Cloud Storage backend, support specifying access tokens
directly, as an alternative to a credentials file. This is useful when
restic is used non-interactively by some other program that is already
authenticated and eliminates the need to store long lived credentials.
The access token is specified in the GOOGLE_ACCESS_TOKEN environment
variable and takes precedence over GOOGLE_APPLICATION_CREDENTIALS.
2020-07-22 16:23:03 +02:00
Alexander Weiss
3b7a3711e6
Add more realistic index benchmarks
...
+ reduce test size of BenchmarkMasterIndexLookupParallel
2020-07-21 07:18:20 +02:00
Michael Eischer
9b0e718852
checker: Test that blob types are not confused
2020-07-20 23:43:47 +02:00
MichaelEischer
82c908871d
Merge pull request #2812 from greatroar/chaining
...
Chaining hash table for repository.Index
2020-07-20 23:29:36 +02:00
Michael Eischer
ddf0b8cd0b
checker: Properly distinguish between data and tree blobs
...
If a data blob and a tree blob with the same ID (= same content) exist,
then the checker did not report a data or tree blob as unused when the
blob of the other type was still in use.
2020-07-20 22:58:39 +02:00
Michael Eischer
2d0c138c9b
checker: Test that check only decodes trees once
...
The `DuplicateTree` flag is necessary to ensure that failures cannot be
swallowed. The old checker implementation ignores errors from LoadTree
if the corresponding tree was already checked.
2020-07-20 22:51:53 +02:00
Michael Eischer
ef325ffc02
checker: Cleanup error handling code
...
This change only moves code around but does not result in any change in
behavior.
2020-07-20 22:51:53 +02:00
Michael Eischer
0f67ae813a
Add changelog
2020-07-20 22:51:53 +02:00
Michael Eischer
7a165f32a9
checker: Traverse trees in depth-first order
...
Backups traverse the file tree in depth-first order and saves trees on
the way back up. This results in tree packs filled in a way comparable
to the reverse Polish notation. In order to check tree blobs in that
order, the treeFilter would have to delay the forwarding of tree nodes
until all children of it are processed which would complicate the
implementation.
Therefore do the next similar thing and traverse the tree in depth-first
order, but process trees already on the way down. The tree blob ids are
added in reverse order to the backlog, which is once again reverted when
removing the ids from the back of the backlog.
2020-07-20 22:51:53 +02:00
Michael Eischer
36c69e3ca7
checker: Unify blobs, processed trees and referenced blobs map
...
The blobRefs map and the processedTrees IDSet are merged to reduce the
memory usage. The blobRefs map now uses separate flags to track blob
usage as data or tree blob. This prevents skipping of trees whose
content is identical to an already processed data blob. A third flag
tracks whether a blob exists or not, which removes the need for the
blobs IDSet.
2020-07-20 22:51:47 +02:00
Michael Eischer
35d8413639
checker: Remove dead index map
2020-07-20 22:37:31 +02:00
Michael Eischer
c66a0e408c
checker: Reduce cost of debug log
...
Avoid duplicate allocation of the Subtree list.
2020-07-20 22:37:31 +02:00
Michael Eischer
70f4c014ef
checker: Decode identical tree nodes only once
...
Even though the checkTreeWorker skips already processed chunks,
filterTrees did queue the same tree blob on every occurence. This
becomes a serious performance bottleneck for larger number of snapshots
that cover mostly the same directories. Therefore decode a tree blob
exactly once.
2020-07-20 22:37:31 +02:00
Michael Eischer
f0d8710611
Add benchmark for checker scaling with snapshot count
2020-07-20 22:37:31 +02:00
MichaelEischer
bd3e280f6d
Merge pull request #2546 from Summerdave/fix-backup-error-code
...
Return an error when errors occured during backup
2020-07-20 22:31:46 +02:00
MichaelEischer
2746dcdb5f
Merge pull request #2845 from restic/update-chunker
...
Update chunker
2020-07-20 22:20:28 +02:00
David Sommer
5729d967f5
backup: Return exit status code 3 when failing to read source data
...
The backup command used to return a zero exit code as long as a snapshot
could be created successfully, even if some of the source files could not
be read (in which case the snapshot would contain the rest of the files).
This made it hard for automation/scripts to detect failures/incomplete
backups by looking at the exit code. Restic now returns the following exit
codes for the backup command:
- 0 when the command was successful
- 1 when there was a fatal error (no snapshot created)
- 3 when some source data could not be read (incomplete snapshot created)
2020-07-20 22:19:01 +02:00
Alexander Neumann
f9f6124558
Update chunker
2020-07-20 22:07:02 +02:00
Alexander Neumann
8074879c5f
Remove 'go generate'
2020-07-19 17:28:42 +02:00
greatroar
7bda28f31f
Chaining hash table for repository.Index
...
These are faster to construct but slower to access. The allocation rate
is halved, the peak memory usage almost halved compared to standard map.
Benchmark results on linux/amd64, -benchtime=3s -count=20:
name old time/op new time/op delta
PackerManager-8 178ms ± 0% 178ms ± 0% ~ (p=0.231 n=20+20)
DecodeIndex-8 4.54s ± 0% 4.30s ± 0% -5.20% (p=0.000 n=18+17)
DecodeIndexParallel-8 4.54s ± 0% 4.30s ± 0% -5.22% (p=0.000 n=19+18)
IndexHasUnknown-8 44.4ns ± 5% 50.5ns ±11% +13.82% (p=0.000 n=19+17)
IndexHasKnown-8 48.3ns ± 0% 51.5ns ±12% +6.68% (p=0.001 n=16+20)
IndexAlloc-8 758ms ± 1% 616ms ± 1% -18.69% (p=0.000 n=19+19)
IndexAllocParallel-8 234ms ± 3% 204ms ± 2% -12.60% (p=0.000 n=20+18)
MasterIndexLookupSingleIndex-8 122ns ± 0% 145ns ± 9% +18.44% (p=0.000 n=14+20)
MasterIndexLookupMultipleIndex-8 369ns ± 2% 429ns ± 8% +16.27% (p=0.000 n=20+20)
MasterIndexLookupSingleIndexUnknown-8 68.4ns ± 5% 74.9ns ±13% +9.47% (p=0.000 n=20+20)
MasterIndexLookupMultipleIndexUnknown-8 315ns ± 3% 369ns ±11% +17.14% (p=0.000 n=20+20)
MasterIndexLookupParallel/known,indices=5-8 743ns ± 1% 816ns ± 2% +9.87% (p=0.000 n=17+17)
MasterIndexLookupParallel/unknown,indices=5-8 238ns ± 1% 260ns ± 2% +9.14% (p=0.000 n=19+20)
MasterIndexLookupParallel/known,indices=10-8 1.01µs ± 3% 1.11µs ± 2% +9.79% (p=0.000 n=19+20)
MasterIndexLookupParallel/unknown,indices=10-8 222ns ± 0% 269ns ± 2% +20.83% (p=0.000 n=16+20)
MasterIndexLookupParallel/known,indices=20-8 1.06µs ± 2% 1.19µs ± 2% +12.95% (p=0.000 n=19+18)
MasterIndexLookupParallel/unknown,indices=20-8 413ns ± 1% 530ns ± 1% +28.19% (p=0.000 n=18+20)
SaveAndEncrypt-8 30.2ms ± 1% 30.4ms ± 0% +0.71% (p=0.000 n=19+19)
LoadTree-8 540µs ± 1% 576µs ± 1% +6.73% (p=0.000 n=20+20)
LoadBlob-8 5.64ms ± 0% 5.64ms ± 0% ~ (p=0.883 n=18+17)
LoadAndDecrypt-8 5.93ms ± 0% 5.95ms ± 1% ~ (p=0.247 n=20+19)
LoadIndex-8 25.1ms ± 0% 24.5ms ± 1% -2.54% (p=0.000 n=18+17)
name old speed new speed delta
PackerManager-8 296MB/s ± 0% 296MB/s ± 0% ~ (p=0.229 n=20+20)
SaveAndEncrypt-8 139MB/s ± 1% 138MB/s ± 0% -0.71% (p=0.000 n=19+19)
LoadBlob-8 177MB/s ± 0% 177MB/s ± 0% ~ (p=0.890 n=18+17)
LoadAndDecrypt-8 169MB/s ± 0% 168MB/s ± 1% ~ (p=0.227 n=20+19)
name old alloc/op new alloc/op delta
PackerManager-8 91.8kB ± 0% 91.8kB ± 0% ~ (p=0.772 n=12+19)
IndexAlloc-8 786MB ± 0% 400MB ± 0% -49.04% (p=0.000 n=20+18)
IndexAllocParallel-8 786MB ± 0% 401MB ± 0% -49.04% (p=0.000 n=19+15)
SaveAndEncrypt-8 21.0MB ± 0% 21.0MB ± 0% +0.00% (p=0.000 n=19+19)
name old allocs/op new allocs/op delta
PackerManager-8 1.41k ± 0% 1.41k ± 0% ~ (all equal)
IndexAlloc-8 977k ± 0% 907k ± 0% -7.18% (p=0.000 n=20+20)
IndexAllocParallel-8 977k ± 0% 907k ± 0% -7.17% (p=0.000 n=19+15)
SaveAndEncrypt-8 73.0 ± 0% 73.0 ± 0% ~ (all equal)
2020-07-19 13:58:22 +02:00
greatroar
255ba83c4b
Parallel index benchmarks + benchmark optimizations
...
createRandomIndex was using the global RNG, which locks on every call
It was also using twice as many random numbers as necessary and doing
a float division in every iteration of the inner loop.
BenchmarkDecodeIndex was using too short an input, especially for a
parallel version. (It may now be using one that is a bit large.)
Results on linux/amd64, -benchtime=3s -count=20:
name old time/op new time/op delta
PackerManager-8 178ms ± 0% 178ms ± 0% ~ (p=0.165 n=20+20)
DecodeIndex-8 13.6µs ± 2% 4539886.8µs ± 0% +33293901.38% (p=0.000 n=20+18)
IndexHasUnknown-8 44.4ns ± 7% 44.4ns ± 5% ~ (p=0.873 n=20+19)
IndexHasKnown-8 49.2ns ± 3% 48.3ns ± 0% -1.86% (p=0.000 n=20+16)
IndexAlloc-8 802ms ± 1% 758ms ± 1% -5.51% (p=0.000 n=20+19)
MasterIndexLookupSingleIndex-8 124ns ± 1% 122ns ± 0% -1.41% (p=0.000 n=20+14)
MasterIndexLookupMultipleIndex-8 373ns ± 2% 369ns ± 2% -1.13% (p=0.001 n=20+20)
MasterIndexLookupSingleIndexUnknown-8 67.8ns ± 3% 68.4ns ± 5% ~ (p=0.753 n=20+20)
MasterIndexLookupMultipleIndexUnknown-8 316ns ± 3% 315ns ± 3% ~ (p=0.846 n=20+20)
SaveAndEncrypt-8 30.5ms ± 1% 30.2ms ± 1% -1.09% (p=0.000 n=19+19)
LoadTree-8 527µs ± 1% 540µs ± 1% +2.37% (p=0.000 n=19+20)
LoadBlob-8 5.65ms ± 0% 5.64ms ± 0% -0.21% (p=0.000 n=19+18)
LoadAndDecrypt-8 7.07ms ± 2% 5.93ms ± 0% -16.15% (p=0.000 n=19+20)
LoadIndex-8 32.1ms ± 2% 25.1ms ± 0% -21.64% (p=0.000 n=20+18)
name old speed new speed delta
PackerManager-8 296MB/s ± 0% 296MB/s ± 0% ~ (p=0.159 n=20+20)
SaveAndEncrypt-8 138MB/s ± 1% 139MB/s ± 1% +1.10% (p=0.000 n=19+19)
LoadBlob-8 177MB/s ± 0% 177MB/s ± 0% +0.21% (p=0.000 n=19+18)
LoadAndDecrypt-8 141MB/s ± 2% 169MB/s ± 0% +19.24% (p=0.000 n=19+20)
name old alloc/op new alloc/op delta
PackerManager-8 91.8kB ± 0% 91.8kB ± 0% ~ (p=0.826 n=19+12)
IndexAlloc-8 786MB ± 0% 786MB ± 0% +0.01% (p=0.000 n=20+20)
SaveAndEncrypt-8 21.0MB ± 0% 21.0MB ± 0% -0.00% (p=0.012 n=20+19)
name old allocs/op new allocs/op delta
PackerManager-8 1.41k ± 0% 1.41k ± 0% ~ (all equal)
IndexAlloc-8 977k ± 0% 977k ± 0% +0.01% (p=0.022 n=20+20)
SaveAndEncrypt-8 73.0 ± 0% 73.0 ± 0% ~ (all equal)
2020-07-19 13:58:05 +02:00
MichaelEischer
7dc200c593
Merge pull request #2832 from alrs/fix-fuse-test-err
...
internal/fuse: fix dropped test error
2020-07-16 22:28:20 +02:00
Lars Lehtonen
9ac90cf5cd
internal/fuse: fix dropped test error
2020-07-12 21:42:31 -07:00
MichaelEischer
b84f5177cb
Merge pull request #2790 from greatroar/fix-quadratic-read
...
Fix quadratic file reading in restic mount
2020-07-12 18:42:14 +02:00
greatroar
4cf1c8e8da
Changelog entry for subquadratic reading in mount
2020-07-12 18:27:16 +02:00
greatroar
58719e1f47
Replace mount's per-file cache by a global LRU cache
2020-07-12 18:27:16 +02:00
greatroar
d42c169458
Fix quadratic file reading in restic mount
2020-07-12 18:27:16 +02:00
rawtaz
8598bb042b
Merge pull request #2769 from darkdragon-001/patch-1
...
Update documentation for environment variables
2020-07-12 00:40:21 +02:00
darkdragon-001
c6b74962df
Update 040_backup.rst
...
Changes proposed in #2763 :
- Adding `RESTIC_CACHE_DIR` environment variables (introduced in #2425 for Unix and #2607 for Mac, Win).
- Adding used system-wide environment variables with links to the corresponding section.
2020-07-12 00:31:56 +02:00
MichaelEischer
2c72924ffb
Merge pull request #2813 from greatroar/encrypt-benchmark
...
Fix repository_test.BenchmarkSaveAndEncrypt
2020-07-05 17:52:10 +02:00
greatroar
02bec13ef2
Fix repository_test.BenchmarkSaveAndEncrypt
...
The benchmark was actually testing the speed of index lookups.
name old time/op new time/op delta
SaveAndEncrypt-8 101ns ± 2% 31505824ns ± 1% +31311591.31% (p=0.000 n=10+10)
name old speed new speed delta
SaveAndEncrypt-8 41.7TB/s ± 2% 0.0TB/s ± 1% -100.00% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
SaveAndEncrypt-8 1.00B ± 0% 20989508.40B ± 0% +2098950740.00% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
SaveAndEncrypt-8 0.00 123.00 ± 0% +Inf% (p=0.000 n=10+9)
(The actual speed is ca. 131MiB/s.)
2020-07-05 17:41:42 +02:00
MichaelEischer
64976b1a4d
Merge pull request #2815 from scelfo/patch-1
...
--hostname flag is deprecated
2020-06-29 22:04:56 +02:00
Tony Scelfo
6a607d6ded
--hostname flag is deprecated
...
Update documentation to follow the deprecation warning "Flag --hostname has been deprecated, use --host" that is given on the latest build(s).
2020-06-29 12:53:57 -06:00
rawtaz
6fedf1a7f4
Merge pull request #2809 from lessfoobar/patch-2
...
Update 080_examples.rst
2020-06-26 00:12:05 +02:00
@lessfoobar
df946fd9f8
Update 080_examples.rst
2020-06-26 00:01:30 +02:00
Jakub Wilk
4e6a9767de
Fix examples of --verbose with argument
2020-06-23 20:22:14 +02:00
rawtaz
1bc80c3c8d
Merge pull request #2802 from andreaso/cachedir-https-links
...
Use https:// for linking CACHEDIR.TAG specs
2020-06-21 20:01:42 +02:00
Andreas Olsson
0fcef2ec23
Use https:// for linking CACHEDIR.TAG specs
...
That site might not have supported https:// when those links were
originally added. It does now.
Also dropping the _spec.html_ ending of the url, there being a `<link
rel="canonical" ...>` tag suggesting that that no longer being the
preferred address.
2020-06-21 19:36:26 +02:00
MichaelEischer
212607dc8a
Merge pull request #2760 from greatroar/backend-benchmark
...
Fix backend benchmarks + a micro-optimization
2020-06-17 23:17:05 +02:00
greatroar
55c3a90a0d
Clarify rclone-over-SSH docs
...
Also added a link to S. Ruderich's blog post explaining append-only
repos using rclone and SSH.
2020-06-17 15:22:20 +02:00
greatroar
190d8e2f51
Flatten backend.LimitedReadCloser structure
...
This inlines the io.LimitedReader into the LimitedReadCloser body to
achieve fewer allocations. Results on linux/amd64:
name old time/op new time/op delta
Backend/BenchmarkLoadPartialFile-8 412µs ± 4% 413µs ± 4% ~ (p=0.634 n=17+17)
Backend/BenchmarkLoadPartialFileOffset-8 455µs ±13% 441µs ±10% ~ (p=0.072 n=20+18)
name old speed new speed delta
Backend/BenchmarkLoadPartialFile-8 10.2GB/s ± 3% 10.2GB/s ± 4% ~ (p=0.817 n=16+17)
Backend/BenchmarkLoadPartialFileOffset-8 9.25GB/s ±12% 9.54GB/s ± 9% ~ (p=0.072 n=20+18)
name old alloc/op new alloc/op delta
Backend/BenchmarkLoadPartialFile-8 888B ± 0% 872B ± 0% -1.80% (p=0.000 n=15+15)
Backend/BenchmarkLoadPartialFileOffset-8 888B ± 0% 872B ± 0% -1.80% (p=0.000 n=15+15)
name old allocs/op new allocs/op delta
Backend/BenchmarkLoadPartialFile-8 18.0 ± 0% 17.0 ± 0% -5.56% (p=0.000 n=15+15)
Backend/BenchmarkLoadPartialFileOffset-8 18.0 ± 0% 17.0 ± 0% -5.56% (p=0.000 n=15+15)
2020-06-17 13:11:45 +02:00
greatroar
f4cd2a7120
Make backend benchmarks fairer by removing checks
...
Checking whether the right data is returned takes up half the time in
some benchmarks. Results for local backend benchmarks on linux/amd64:
name old time/op new time/op delta
Backend/BenchmarkLoadFile-8 4.89ms ± 0% 2.72ms ± 1% -44.26% (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFile-8 936µs ± 6% 439µs ±15% -53.07% (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFileOffset-8 940µs ± 1% 456µs ±10% -51.50% (p=0.008 n=5+5)
Backend/BenchmarkSave-8 23.9ms ±14% 24.8ms ±41% ~ (p=0.690 n=5+5)
name old speed new speed delta
Backend/BenchmarkLoadFile-8 3.43GB/s ± 0% 6.16GB/s ± 1% +79.40% (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFile-8 4.48GB/s ± 6% 9.63GB/s ±14% +114.78% (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFileOffset-8 4.46GB/s ± 1% 9.22GB/s ±10% +106.74% (p=0.008 n=5+5)
Backend/BenchmarkSave-8 706MB/s ±13% 698MB/s ±31% ~ (p=0.690 n=5+5)
2020-06-17 13:11:45 +02:00
MichaelEischer
aba270df7e
Merge pull request #2786 from aawsome/memory-list-blobs
...
Loop over index files for 'list blobs'
2020-06-15 22:27:29 +02:00
Alexander Weiss
b5543cff5d
Loop over index files for 'list blobs'
...
=> reduces memory consumption a lot!
2020-06-14 17:25:34 +02:00
Alexander Neumann
285b5236c2
Merge pull request #2684 from MichaelEischer/cleanup-output
...
Cleanup command output code
2020-06-14 11:25:40 +02:00
Michael Eischer
bb1e258bb7
cat/stats: Get stdout from globalOptions
2020-06-14 11:08:11 +02:00
Michael Eischer
182655bc88
Replace fmt.Printf/Println/Fprintf with wrapper functions
...
cmd/restic/globals.go already provides Printf, Println and Warnf wrapper
which get their output streams from the globalOptions object. This
allows for stream replacements when testing.
2020-06-14 11:08:11 +02:00
MichaelEischer
74bc7141c1
Merge pull request #2781 from aawsome/reduce-index-memory
...
Reduce index memory
2020-06-14 10:58:28 +02:00
Alexander Weiss
1361341c58
don't save duplicate packIDs when using internal/repository/Index.Store
2020-06-14 07:56:24 +02:00
Alexander Weiss
ce4a2f4ca6
save packIDs and duplicates separately
...
A side remark to the definition of Index.blob:
Another possibility would have been to use:
blob map[restic.BlobHandle]*indexEntry
This would have led to the following sizes:
key: 32 + 1 = 33 bytes
value: 8 bytes
indexEntry: 8 + 4 + 4 = 16 bytes
each packID: 32 bytes
To save N index entries, we would therefore have needed:
N * OF * (33 + 8) bytes + N * 16 + N * 32 bytes / BP = N * 82 bytes
More precicely, using a pointer instead of a direct entry is the better memory choice if:
OF * 8 bytes + entrysize < OF * entrysize <=> entrysize > 8 bytes * OF/(OF-1)
Under the assumption of OF=1.5, this means using pointers would have been the better choice
if sizeof(indexEntry) > 24 bytes.
2020-06-14 07:56:21 +02:00
Alexander Weiss
cf979e2b81
make offset and length uint32
2020-06-14 07:50:19 +02:00
Michael Eischer
d92e2c5769
simplify index code
2020-06-14 07:50:19 +02:00
Alexander Weiss
7419844885
add changelog, benchmark, memory calculation
2020-06-14 07:50:15 +02:00
rawtaz
1d66bb9e62
Merge pull request #2784 from MichaelEischer/minio-sha256-changelog
...
Add changelog for #2709
2020-06-13 18:33:54 +02:00
Michael Eischer
0b2c31b05b
Add changelog for #2709
2020-06-13 18:13:36 +02:00
Alexander Weiss
d3c59d18e5
Fix inconsistency of saving/loading config file
...
Fix saving/loading config file: Always set ID to a zero ID.
2020-06-13 16:30:23 +02:00
MichaelEischer
dd7b4f54f5
Merge pull request #2709 from greatroar/minio-sha256
...
Use Minio's optimized SHA-256
2020-06-12 23:32:58 +02:00
MichaelEischer
6896c6449b
Merge pull request #2779 from greatroar/archiver-comment
...
Fix up comment on archiver.BlobSaver.Save
2020-06-12 23:04:21 +02:00
MichaelEischer
735a8074d5
Merge pull request #2773 from aawsome/index-uploads+knownblobs
...
Fix non-intuitive repo behavior
2020-06-12 22:41:04 +02:00
Alexander Weiss
70347e95d5
disable index uploads for prune command
...
+ modifications of changelog
2020-06-12 09:24:38 +02:00
greatroar
0fa3091c78
Fix up comment on archiver.BlobSaver.Save
2020-06-11 13:40:30 +02:00
Alexander Weiss
91906911b0
Fix non-intuitive repository behavior
...
- The SaveBlob method now checks for duplicates.
- Moves handling of pending blobs to MasterIndex.
-> also cleans up pending index entries when they are saved in the index
-> when using SaveBlob no need to care about index any longer
- Always check for full index and save it when storing packs.
-> removes the need of an index uploader
-> also removes the verbose "uploaded intermediate index" messages
- The Flush method now also saves the index
- Fix race condition when checking and saving full/non-finalized indexes
2020-06-11 13:05:23 +02:00
MichaelEischer
fae7f78057
Merge pull request #2755 from greatroar/solaris-workaround
...
Remove Solaris build workaround in integration tests
2020-06-10 23:30:51 +02:00
MichaelEischer
ac9ec4b990
Merge pull request #2778 from MichaelEischer/fix-unlock-hint
...
Revive hint to the unlock command if a repository is locked
2020-06-10 21:26:38 +02:00
Michael Eischer
087c770161
Revive hint to the unlock command if a repository is locked
...
errors.Fatalf wraps a error and just keeps an error message as a string.
This prevents the `restic.IsAlreadyLocked(err)` check from working as
the error is no longer an ErrAlreadyLocked.
Just add an additional remark to the error using `errors.WithMessage`.
2020-06-10 20:58:56 +02:00
MichaelEischer
6856d1e422
Merge pull request #2749 from aawsome/fix-fullindex
...
Change condition for full index
2020-06-10 20:40:19 +02:00
Alexander Weiss
8c1261ff02
changed condition for full index
2020-06-07 22:00:49 +02:00
rawtaz
26704be17f
Merge pull request #2776 from nunoperalta/patch-1
...
Fix #2774 - make unable to umount message clearer
2020-06-07 17:19:50 +02:00
nunoperalta
2c3360db98
Fix #2774 - make unable to umount message clearer
2020-06-07 15:57:39 +01:00
MichaelEischer
cba6ad8d8e
Merge pull request #2711 from greatroar/update-x-net
...
Update golang.org/x/net for fewer allocations in http2
2020-05-25 20:43:29 +02:00
greatroar
2a3312ac35
Remove Solaris build workaround in integration tests
...
This was introduced in #1821 because of issues with Go 1.9, but that
compiler version is no longer supported.
2020-05-25 10:33:36 +02:00
greatroar
c35c4e0cbf
Update golang.org/x/net for fewer allocations in http2
...
name old allocs/op new allocs/op delta
BackendREST/BenchmarkLoadFile-8 2.15k ± 0% 2.14k ± 0% -0.27% (p=0.002 n=20+20)
BackendREST/BenchmarkLoadPartialFile-8 587 ± 0% 582 ± 0% -0.88% (p=0.000 n=20+20)
BackendREST/BenchmarkLoadPartialFileOffset-8 584 ± 0% 579 ± 0% -0.90% (p=0.000 n=19+16)
The throughput isn't significantly higher or lower.
2020-05-24 16:39:17 +02:00
MichaelEischer
84475aa3a8
Merge pull request #2730 from greatroar/mount-build-tags
...
Simplify build tags for restic mount
2020-05-23 20:33:43 +02:00
MichaelEischer
f12f9ae240
Merge pull request #2741 from tbm/typos
...
Fix typos
2020-05-21 23:40:12 +02:00
Martin Michlmayr
5cc1760fdf
Fix typos
2020-05-16 14:05:26 +08:00
MichaelEischer
32ac5486e9
Merge pull request #2732 from restic/rawtaz-update-contributing
...
Reword parts of Providing Patches in CONTRIBUTING.md
2020-05-12 21:37:08 +02:00
rawtaz
c4336978eb
Reword parts of Providing Patches in CONTRIBUTION.md
...
Primarily makes the request that contributors create a new branch for their changes stand out more.
2020-05-12 14:03:57 +02:00
greatroar
649cbec6c5
Simplify build tags for restic mount
...
This command can only be built on Darwin, FreeBSD and Linux
(and if we upgrade bazil.org/fuse, only FreeBSD and Linux:
https://github.com/bazil/fuse/issues/224 ).
Listing the few supported operating systems explicitly here makes
porting restic to new platforms easier.
2020-05-12 11:30:41 +02:00
MichaelEischer
b17bd7f860
Merge pull request #2719 from greatroar/update-x-crypto
...
Update x/crypto to stop poly1305 segfaulting on ARM
2020-05-09 00:02:18 +02:00
greatroar
68f1e9c524
Update x/crypto to fix poly1305 on ARM
...
The broken poly1305 implementation for arm64 was removed, fixing the
segfault issue on arm64 (gh-2618). The version for amd64 has improved
performance, which shows up in the internal/repository benchmark:
name old speed new speed delta
SaveAndEncrypt-8 110MB/s ± 2% 113MB/s ± 1% +2.24% (p=0.000 n=20+20)
2020-05-05 11:05:35 +02:00
MichaelEischer
1ee2306033
Merge pull request #2717 from MichaelEischer/fix-flaky-password-prompt
...
backup: Avoid race between password prompt and open repository message
2020-05-01 23:58:55 +02:00
Michael Eischer
c882a92cd6
backup: Avoid race between password prompt and open repository message
...
`term.Print` sends the output via a channel to a goroutine which
actually prints the message. This may race with the password prompt
printed by `OpenRepository` resulting in a missing prompt.
2020-05-01 23:42:16 +02:00
MichaelEischer
f54db5d796
Merge pull request #2713 from greatroar/unused
...
Move Index.FindBlob to tests
2020-05-01 20:41:59 +02:00
MichaelEischer
843e7f404e
Merge pull request #2716 from MichaelEischer/revert-apfs-test-workaround
...
Revert "Darwin test fix: allow 1μs timestamp difference"
2020-05-01 18:13:38 +02:00
Michael Eischer
d465b5b9ad
Revert "Darwin test fix: allow 1μs timestamp difference"
...
This reverts commit f3016a9096
.
2020-05-01 17:11:30 +02:00
greatroar
9f7cd69f13
Move Index.FindBlob to tests
2020-04-29 10:57:01 +02:00
greatroar
f97a680887
Fix repository benchmarks
...
BenchmarkLoad{AndDecrypt,Blob} were spending between 38% and 50% of
their time measuring SHA-256 performance in their checks.
2020-04-28 07:57:29 +02:00
greatroar
42a3db05b0
Use Minio's optimized SHA-256
...
internal/repository benchmarks on an Intel i7-3770k:
name old speed new speed delta
PackerManager-8 209MB/s ± 1% 291MB/s ± 1% +38.94% (p=0.008 n=5+5)
SaveAndEncrypt-8 112MB/s ± 1% 135MB/s ± 1% +20.25% (p=0.008 n=5+5)
2020-04-28 07:57:18 +02:00
MichaelEischer
070d43e290
Merge pull request #2704 from azak-azkaran/patch-1
...
Update 030_preparing_a_new_repo.rst
2020-04-23 20:48:34 +02:00
MichaelEischer
d4bd32a37e
Merge pull request #2640 from greatroar/simplify-loadblob-usage
...
Simplify Repository.LoadBlob usage
2020-04-23 20:23:35 +02:00
greatroar
e7d7b85d59
Merge Repository.{LoadBlob,loadBlob}
...
Pushing the allocation logic down into the former loadBlob body means
that fewer allocations have to be performed:
name old time/op new time/op delta
LoadTree-8 478µs ± 1% 481µs ± 2% ~ (p=0.315 n=9+10)
LoadBlob-8 11.6ms ± 1% 11.6ms ± 2% ~ (p=0.393 n=10+10)
LoadAndDecrypt-8 13.3ms ± 3% 13.3ms ± 3% ~ (p=0.905 n=10+9)
LoadIndex-8 33.6ms ± 2% 33.2ms ± 1% -1.15% (p=0.028 n=10+9)
name old alloc/op new alloc/op delta
LoadTree-8 41.2kB ± 0% 41.1kB ± 0% -0.23% (p=0.000 n=10+10)
LoadBlob-8 2.28kB ± 0% 2.18kB ± 0% -4.21% (p=0.000 n=10+10)
LoadAndDecrypt-8 2.10MB ± 0% 2.10MB ± 0% ~ (all equal)
LoadIndex-8 5.22MB ± 0% 5.22MB ± 0% ~ (p=0.631 n=10+10)
name old allocs/op new allocs/op delta
LoadTree-8 652 ± 0% 651 ± 0% -0.15% (p=0.000 n=10+10)
LoadBlob-8 24.0 ± 0% 23.0 ± 0% -4.17% (p=0.000 n=10+10)
LoadAndDecrypt-8 30.0 ± 0% 30.0 ± 0% ~ (all equal)
LoadIndex-8 30.2k ± 0% 30.2k ± 0% ~ (p=0.610 n=10+10)
name old speed new speed delta
LoadBlob-8 86.4MB/s ± 1% 85.9MB/s ± 2% ~ (p=0.393 n=10+10)
LoadAndDecrypt-8 75.4MB/s ± 3% 75.4MB/s ± 3% ~ (p=0.858 n=10+9)
2020-04-23 10:04:20 +02:00
greatroar
be5a0ff59f
Centralize buffer allocation and size checking in Repository.LoadBlob
...
Benchmark results for internal/repository:
name old time/op new time/op delta
LoadTree-8 479µs ± 2% 478µs ± 1% ~ (p=0.780 n=10+9)
LoadBlob-8 11.6ms ± 2% 11.6ms ± 1% ~ (p=0.631 n=10+10)
LoadAndDecrypt-8 13.2ms ± 2% 13.3ms ± 3% ~ (p=0.631 n=10+10)
name old alloc/op new alloc/op delta
LoadTree-8 41.2kB ± 0% 41.2kB ± 0% ~ (all equal)
LoadBlob-8 2.28kB ± 0% 2.28kB ± 0% ~ (all equal)
LoadAndDecrypt-8 2.10MB ± 0% 2.10MB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
LoadTree-8 652 ± 0% 652 ± 0% ~ (all equal)
LoadBlob-8 24.0 ± 0% 24.0 ± 0% ~ (all equal)
LoadAndDecrypt-8 30.0 ± 0% 30.0 ± 0% ~ (all equal)
name old speed new speed delta
LoadBlob-8 86.2MB/s ± 2% 86.4MB/s ± 1% ~ (p=0.594 n=10+10)
LoadAndDecrypt-8 75.7MB/s ± 2% 75.4MB/s ± 3% ~ (p=0.617 n=10+10)
2020-04-23 10:04:20 +02:00
azak-azkaran
c5100d5632
Update 030_preparing_a_new_repo.rst
...
fixed markdown for wasabi region link
2020-04-23 09:24:19 +02:00
MichaelEischer
956a1b0f96
Merge pull request #2294 from BenWiederhake/debugsyms
...
build: Preserve debug symbols in debug build
2020-04-19 17:26:03 +02:00
Ben Wiederhake
fab626a3df
build: Preserve debug symbols in debug and profile build
...
Signed-off-by: Ben Wiederhake <BenWiederhake.GitHub@gmx.de>
2020-04-19 14:48:40 +02:00
MichaelEischer
4f00564574
Merge pull request #2621 from greatroar/fixes
...
Some small fixes
2020-04-18 18:07:08 +02:00
MichaelEischer
f77477129f
Merge pull request #2584 from greatroar/mount-cache-uid-gid
...
Cache uid and gid for top directories in internal/fuse
2020-04-18 17:45:14 +02:00
greatroar
2e31120f89
Remove unused argument to restic.fakeFile
2020-04-18 17:40:13 +02:00
greatroar
8fb2c0d3c1
Typo in crypto test name
2020-04-18 17:39:06 +02:00
greatroar
072cf7b02d
Fix debug messages in internal/fuse
2020-04-18 17:39:06 +02:00
greatroar
df66daa5c9
Fix context usage in backend tests
...
Found by go vet. This is also the only complaint is has.
2020-04-18 17:39:06 +02:00
MichaelEischer
9790d8ce1c
Merge pull request #2668 from MichaelEischer/fix-stats-blobs-crash
...
stats: Fix crash in blobs-per-file mode on missing blob
2020-04-18 17:21:26 +02:00
MichaelEischer
16710454f4
Merge pull request #2628 from MichaelEischer/one-element-pack-lists
...
cache: Don't sort one element pack lists
2020-04-18 17:09:06 +02:00
MichaelEischer
08ec6c9f17
Merge pull request #2677 from MichaelEischer/complain-about-invalid-indexes
...
rebuild_index: Report invalid packs that were ignored
2020-04-18 16:57:50 +02:00
MichaelEischer
7910ff4c0e
Merge pull request #2648 from nairb774/iowritestring
...
termstatus: Use io.WriteString to output messages.
2020-04-18 13:48:42 +02:00
MichaelEischer
c4da9d1e90
Merge pull request #2638 from greatroar/no-close-in-packer
...
Don't Close in Packer.Finalize
2020-04-18 13:07:21 +02:00
rawtaz
3ed61987a2
Merge pull request #2695 from restic/rawtaz-doc-exclude-file-tilde
...
doc: Add note on tilde expansion in exclude files
2020-04-18 13:01:17 +02:00
rawtaz
9dba7a2577
doc: Add note on tilde expansion in exclude files
...
Explains to the reader that tilde expansion does not work in exclude files, and that they should instead use the $HOME variable.
2020-04-18 12:47:47 +02:00
MichaelEischer
a1352906e2
Merge pull request #2622 from greatroar/optimize-packer-manager
...
Fix PackerManager benchmark and optimize hashing.Writer
2020-04-18 12:46:34 +02:00
MichaelEischer
b7c0d4d8bf
Merge pull request #2644 from greatroar/signal-notify-buffered
...
Make all signal.Notify channels buffered
2020-04-18 11:31:02 +02:00
MichaelEischer
f033850aa0
Merge pull request #2692 from MichaelEischer/fix-archiveraborttest
...
archiver: Fix race condition triggered by TestArchiverAbortEarlyOnError
2020-04-13 18:36:48 +02:00
Michael Eischer
bdf7ba20cb
archiver: Fix race condition triggered by TestArchiverAbortEarlyOnError
...
The Save methods of the BlobSaver, FileSaver and TreeSaver return early
on when the archiver is stopped due to an error. For that they select on
both the tomb.Dying() and context.Done() channels, which can lead to a
race condition when the tomb is killed due to an error: The tomb first
closes its Dying channel before canceling all child contexts.
Archiver.SaveDir only aborts its execution once the context was
canceled. When the tomb killing is paused between closing its Dying
channel and canceling the child contexts, this lets the
FileSaver/TreeSaver.Save methods return immediately, however, ScanDir
still reads further files causing the test case to fail.
As a killed tomb always cancels all child contexts and as the Savers
always use a context bound to the tomb, it is sufficient to just use
context.Done() as escape hatch in the Save functions. This fixes the
mismatch between SaveDir and Save.
Adjust the tests to use contexts bound to the tomb for all interactions
with the Savers.
2020-04-13 18:23:17 +02:00
rawtaz
3ee6b8ec63
Merge pull request #2689 from MichaelEischer/fix-background-hang
...
Fix shutdown hang when restic is started as background job
2020-04-12 22:39:44 +02:00
Michael Eischer
4a400f94bb
Add changelog
2020-04-12 22:27:09 +02:00
Michael Eischer
1a1c572bac
Fix shutdown hang when restic is started as background job
...
restic uses a cleanup hook to ensure that it restores the terminal
configuration to a sane state, when restic is interrupted while reading
a password from the terminal. However, this causes a problem, when
restic runs in a background job, as reconfiguring a terminal will cause
a SIGTTOU to be sent to restic pausing it. Therefore, restic seems to
hang on shutdown when it was running in the background.
This commit changes the behavior to only restore the terminal
configuration if restic was interrupted while reading a password from
the terminal. As reading a password from the terminal requires that
restic is in the foreground, this should avoid restic getting stopped.
Fixes #2298
Issue introduced in #402
2020-04-12 22:27:09 +02:00
rawtaz
5a7c27ddb6
Merge pull request #2681 from MichaelEischer/optimize-debug
...
Reduce memory usage and startup time of debug command
2020-04-04 00:08:14 +02:00
Michael Eischer
fb842759fc
debug: don't load the repository index
...
The existing commands don't need a loaded repository index which can
take several minutes to load on larger repositories.
2020-04-04 00:01:01 +02:00
Michael Eischer
7aa2f8a61e
debug: get stdout/stderr from gopts/globalOptions
2020-04-04 00:01:01 +02:00
Michael Eischer
08bf3bae79
debug: explicitly pass stdout to dump functions
2020-04-03 23:32:44 +02:00
rawtaz
e7b741b2d7
Merge pull request #2682 from MichaelEischer/cleanup-cli-paramter-names
...
Cleanup CLI parameter names for backup / global flags
2020-04-03 21:35:34 +02:00
Michael Eischer
6bee62e346
Update doc excerpts for `--help`
...
This adds some previously missing changes and the new paramters names
from the previous commit.
2020-04-03 19:49:06 +02:00
Michael Eischer
bc74cd3ae5
backup/global: Use proper name for command line argument parameters
...
Several paramters printed a generic "string" or "stringArray" name.
2020-04-03 19:49:04 +02:00
Michael Eischer
90243ed1c4
rebuild_index: Report invalid packs that were ignored
2020-04-02 22:38:31 +02:00
Michael Eischer
0ce81d88b6
stats: Fix crash in blobs-per-file mode on missing blob
...
In a damaged repository with a missing blob, the error message tried to
dereference the subtreeID field of the current node, which is a file
however. Said field is set to nil for a file thus causing a segfault
when dereferenced.
Fix this by using the actual parentTreeID.
2020-03-27 22:17:54 +01:00
rawtaz
c03bc88b29
Merge pull request #2669 from MichaelEischer/doc-cifs-linux-bug
...
doc: Warn about compatibility issues with CIFS and restic
2020-03-26 23:01:06 +01:00
Brian Atkinson
b8da7b1f4d
termstatus: Use io.WriteString to output messages.
...
The previous implementation was repeating the implementation that is
found inside of io.WriteString. Simplify by making use of the stdlib's
implementation.
2020-03-26 14:55:00 -07:00
Michael Eischer
f1b4d97945
doc: Warn about compatibility issues with CIFS and restic
...
On Linux CIFS (SMB) seems to be incompatible with the async preemption
implementation of Go 1.14. CIFS seems not to restart syscalls (open,
read, chmod, readdir, ...) as expected by Go, which sets SA_RESTART for
its signal handler to have syscalls restarted automatically. This leads
to Go passing up lots of EINTR return codes to restic.
See https://github.com/restic/restic/issues/2659 for a detailed explanation.
2020-03-26 21:52:37 +01:00
Peter Schultz
90fc639a67
Allow specifying user and host when adding keys
...
The username and hostname for new keys can be specified with the new
--user and --host flags, respectively. The flags are used only by the
`key add` command and are otherwise ignored.
This allows adding keys with for a desired user and host without having
to run restic as that particular user on that particular host, making
automated key management easier.
Co-authored-by: James TD Smith <ahktenzero@mohorovi.cc>
2020-03-23 13:11:10 +01:00
rawtaz
2b5a6d255a
Merge pull request #2660 from restic/rawtaz-delete-old-issue-template
...
Delete ISSUE_TEMPLATE.md (not used anymore)
2020-03-21 20:48:39 +01:00
rawtaz
f004dbe605
Delete ISSUE_TEMPLATE.md (not used anymore)
...
Nowadays the ISSUE_TEMPLATE/ directory and its files are used for this feature.
2020-03-21 20:30:19 +01:00
rawtaz
9efbe98879
Merge pull request #2623 from alrs/internal-restic-err-before-close
...
internal/restic: close os.File after checking for error
2020-03-18 22:23:20 +01:00
rawtaz
7d9300efca
Merge pull request #2637 from greatroar/unused
...
Remove Go 1.5 compatibility code
2020-03-18 22:22:15 +01:00
rawtaz
c38aaaa768
Merge pull request #2652 from greatroar/upgrade-fuse
...
Upgrade bazil.org/fuse and remove restic mount --allow-root
2020-03-18 22:21:50 +01:00
greatroar
8941041355
Upgrade bazil.org/fuse to version that fixes FreeBSD opBmap bug
2020-03-18 10:00:39 +01:00
greatroar
18fee4806f
Remove broken --allow-root from restic mount
2020-03-17 23:35:06 +01:00
greatroar
47d4d5bf1b
Make all signal.Notify channels buffered
2020-03-12 20:59:39 +01:00
greatroar
74a64c47e4
Move testing logic to test file in internal/pack
2020-03-09 14:32:28 +01:00
greatroar
a23e9c86ba
Remove closing logic from Packer.Finalize
...
The method only ever receives *hashing.Writers, which don't implement
io.Closer. These come from packerManager.findPacker and have their
actual writers closed in Repository.savePacker. Moving the closing logic
to hashing.Writer results in "file already closed" errors.
2020-03-09 14:31:45 +01:00
greatroar
4de12bf593
Remove restic.RandReader
...
math/rand.Rand has implemented Reader since Go 1.6. The repacking tests
are not deterministic, but they weren't before, either.
2020-03-09 10:00:28 +01:00
rawtaz
c542a509f0
Merge pull request #2633 from greatroar/fix-ssh-commandline
...
Revert "Put host last in SSH command line"
2020-03-08 17:29:41 +01:00
greatroar
8cf3bb8737
Revert "Put host last in SSH command line"
...
This reverts commit e1969d1e33
.
2020-03-08 16:45:33 +01:00
Michael Eischer
b46cc6d57e
repository: Don't sort one element pack lists
...
When loading a blob, restic first looks up pack files containing the
blob. To avoid unnecessary work an already cached pack file is preferred.
However, if there is only a single pack file to choose from (which is
the normal case) sorting the one-element list won't change anything.
Therefore avoid the unnecessary cache check in that case.
2020-03-07 10:26:06 +01:00
Lars Lehtonen
4a2156d3f0
internal/restic: close os.File after checking for error
2020-03-05 16:22:46 -08:00
greatroar
41fee11f66
Micro-optimization for hashing.Writer/PackerManager
...
name old time/op new time/op delta
PackerManager-8 247ms ± 1% 246ms ± 1% -0.43% (p=0.001 n=18+18)
name old speed new speed delta
PackerManager-8 213MB/s ± 1% 214MB/s ± 1% +0.43% (p=0.001 n=18+18)
name old alloc/op new alloc/op delta
PackerManager-8 92.2kB ± 0% 91.5kB ± 0% -0.82% (p=0.000 n=19+20)
name old allocs/op new allocs/op delta
PackerManager-8 1.43k ± 0% 1.41k ± 0% -1.67% (p=0.000 n=20+20)
2020-03-05 22:30:04 +01:00
greatroar
b592614061
Improve PackerManager benchmark
...
The previous benchmark spent much of its time allocating RNGs and
generating too many random numbers. It now spends 90% of its time
hashing and half of the rest writing to files.
name old time/op new time/op delta
PackerManager-8 319ms ± 1% 247ms ± 1% -22.48% (p=0.000 n=20+18)
name old speed new speed delta
PackerManager-8 143MB/s ± 1% 213MB/s ± 1% +48.63% (p=0.000 n=10+18)
name old alloc/op new alloc/op delta
PackerManager-8 635kB ± 0% 92kB ± 0% -85.48% (p=0.000 n=10+19)
name old allocs/op new allocs/op delta
PackerManager-8 1.64k ± 0% 1.43k ± 0% -12.76% (p=0.000 n=10+20)
2020-03-05 22:30:03 +01:00
greatroar
b7c3039eb2
Remove Go 1.5 compatibility code from PackerManager benchmark
...
This alone is enough to speed up the benchmark by ~10%.
2020-03-05 22:29:06 +01:00
Alexander Neumann
a307797c11
Merge pull request #2612 from restic/rawtaz-maintainer-pr
...
Add notes about maintainer edit access in PRs
2020-03-01 21:24:34 +01:00
rawtaz
a03f107144
Add note about allowing maintainers edits in PRs.
2020-03-01 21:12:11 +01:00
rawtaz
a141ab1bda
Add maintaner edit checkbox to PR template
2020-03-01 21:05:52 +01:00
Alexander Neumann
52abec967f
Merge pull request #2605 from middelink/fix-2604
...
Fix running tests on a SELinux enabled system
2020-03-01 20:25:28 +01:00
Pauline Middelink
2828a9c2b0
Fix running tests on a SELinux enabled system
...
Archivers TestMetadataChanged incorrectly clears the Extended Attributes
from the expected metadata of the temporary file. This is incorrect as on
SELinux enabled filesystem, as the kernel will automaticly add a SElinux
label. However, since ExtendedAttributes{} != ExtendedAttributes{nil} we
still need to clear them if there are no attributes found.
2020-03-01 20:23:22 +01:00
rawtaz
cec7d581f3
Merge pull request #2610 from restic/remove-vendor
...
Remove vendored dependencies
2020-03-01 20:22:45 +01:00
Alexander Neumann
12aa1e61da
CI: Enable Go Module Proxy
2020-03-01 19:58:20 +01:00
Alexander Neumann
95da6c1c1d
Merge pull request #2589 from greatroar/no-stable-sort
...
Replace sort.Stable by sort.Strings
2020-03-01 19:40:28 +01:00
Alexander Neumann
0c03a80fc4
Merge pull request #2592 from greatroar/sftp-ipv6
...
Support IPv6 in SFTP backend
2020-03-01 19:38:44 +01:00
Alexander Neumann
b1c77172c2
Add entry to changelog
2020-03-01 19:32:35 +01:00
Alexander Neumann
c0373cd307
Remove -mod=vendor from all documentation and code
2020-03-01 19:32:35 +01:00
Alexander Neumann
28121090c2
Remove vendor from CI tests
2020-03-01 19:32:35 +01:00
Alexander Neumann
44a57d66c3
Remove vendor from build scripts
2020-03-01 11:30:02 +01:00
Alexander Neumann
266f9dbe16
Remove vendor dir
2020-03-01 11:20:42 +01:00
greatroar
60e4a88f17
Changelog entry for SFTP w/ IPv6 addresses
2020-03-01 11:04:06 +01:00
Alexander Neumann
c50f91b7f9
Merge pull request #2602 from greatroar/no-bufpool
...
Remove sync.Pool from internal/repository
2020-03-01 10:50:57 +01:00
Alexander Neumann
e851d29565
Merge pull request #2608 from greatroar/simplify-termstatus
...
Simplify termstatus
2020-03-01 10:48:29 +01:00
Alexander Neumann
b67b7ebfe6
Merge pull request #2583 from greatroar/unused
...
Remove some unused or duplicated code
2020-03-01 10:46:17 +01:00
greatroar
751eba0e68
Remove unnecessary pipe checks in termstatus
...
canUpdateStatus has already determined that the file descriptor is not a
pipe.
2020-02-29 18:03:49 +01:00
greatroar
7447c44484
Use golang.org/x/sys/windows in termstatus
...
Some functionality is missing, but at least the types are all defined.
Replaced short, word, dword by their Go names to match the x/sys
convention.
2020-02-29 18:03:49 +01:00
greatroar
c8a672fa29
Remove code copy-pasted from x/crypto/ssh/terminal
2020-02-29 18:03:49 +01:00
greatroar
863ba76494
Drop mattn/go-isatty in favor of crypto/ssh/terminal
2020-02-29 18:03:47 +01:00
greatroar
8526cc6647
Remove sync.Pool from internal/repository
...
The pool was used improperly, causing more allocations to be
performed than without it.
name old time/op new time/op delta
SaveAndEncrypt-8 36.8ms ± 2% 36.9ms ± 2% ~ (p=0.218 n=10+10)
name old speed new speed delta
SaveAndEncrypt-8 114MB/s ± 2% 114MB/s ± 2% ~ (p=0.218 n=10+10)
name old alloc/op new alloc/op delta
SaveAndEncrypt-8 21.1MB ± 0% 21.0MB ± 0% -0.44% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
SaveAndEncrypt-8 79.0 ± 0% 77.0 ± 0% -2.53% (p=0.000 n=10+10)
2020-02-29 17:54:46 +01:00
rawtaz
694b7a17e7
Merge pull request #2607 from greatroar/cachedir
...
Honor RESTIC_CACHE_DIR on Mac and Windows
2020-02-28 20:41:32 +01:00
greatroar
5cd0bce452
Honor RESTIC_CACHE_DIR on Mac and Windows
2020-02-28 15:44:32 +01:00
rawtaz
58bd165253
Merge pull request #2581 from aawsome/multiple-hostnames
...
Allow multiple hostnames tags
2020-02-27 08:35:23 +01:00
rawtaz
65d3fb6b33
Merge pull request #2603 from greatroar/restorer-waitgroup
...
Fix unsafe sync.WaitGroup usage in restorer.fileRestorer
2020-02-27 00:30:59 +01:00
rawtaz
f165048172
Merge pull request #2582 from greatroar/no-go1.9
...
Remove Go 1.9 compatibility code
2020-02-27 00:15:06 +01:00
greatroar
de5516a90e
Fix sync.WaitGroup usage in restorer.fileRestorer
2020-02-27 00:07:49 +01:00
greatroar
4f6fd9fb98
Remove remnant of Go 1.9 compatibility code from tests
2020-02-26 22:23:38 +01:00
Alexander Weiss
9a9101d144
Support specifying multiple host flags for various commands
...
The `dump`, `find`, `forget`, `ls`, `mount`, `restore`, `snapshots`,
`stats` and `tag` commands will now take into account multiple
`--host` and `-H` flags.
2020-02-26 22:17:59 +01:00
rawtaz
616f9499ae
Merge pull request #2600 from restic/update-go
...
Update Go version to >= 1.11, add Go 1.14
2020-02-26 21:29:30 +01:00
Alexander Neumann
a40ac37550
Add changelog file
2020-02-26 20:55:12 +01:00
Alexander Neumann
99fd80a585
Remove all workarounds for Go < 1.11
2020-02-26 20:35:13 +01:00
Alexander Neumann
2464f7c4d1
Update Go versions, drop Go 1.10
2020-02-26 20:35:13 +01:00
rawtaz
b5c7778428
Merge pull request #2195 from ifedorenko/out-of-order-restore-no-progress
...
restorer: allow writing target file blobs out of order
2020-02-26 20:07:51 +01:00
Igor Fedorenko
c52198d12c
restorer: go mod vendor; go mod tidy
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2020-02-26 16:15:04 +01:00
Igor Fedorenko
f17ffa0283
restorer: Allow writing target file blobs out of order
...
Much simpler implementation that guarantees each required pack
is downloaded only once (and hence does not need to manage
pack cache). Also improves large file restore performance.
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2020-02-26 16:14:45 +01:00
greatroar
5e2afd91e7
Assert that archiver.Tree implements fmt.Stringer
2020-02-26 11:05:38 +01:00
greatroar
79b882e901
Merge duplicated readdir functionality
...
internal/archiver.readdir and internal/fs.ReadDir were unused.
internal/fs.ReadDirNames and internal/archiver.readdirnames were doing
nearly the same thing, except one sorted its output and opened with
fs.O_NOFOLLOW. Both were only used in internal/archiver.
2020-02-26 11:05:38 +01:00
greatroar
1b502fa9ef
Cache uid and gid for top directories in internal/fuse
2020-02-24 10:46:09 +01:00
greatroar
e1969d1e33
Put host last in SSH command line
...
This is how the SSH manpage says the command line should look, and the
"--" prevents mistakes in hostnames from being interpreted as options.
2020-02-19 15:53:20 +01:00
greatroar
6ac6bca7a1
Support IPv6 in SFTP backend
...
The previous code was doing its own hostname:port splitting, which
caused IPv6 addresses to be misinterpreted.
2020-02-19 15:42:12 +01:00
greatroar
3a6feb0596
Replace sort.Stable by sort.Strings
...
Calling the slow, O(n lg² n) sort.Stable is equivalent to sort.Strings
for a slice of unique strings.
2020-02-18 19:41:06 +01:00
greatroar
2f8aa2ce30
Remove unused fs.FS from archiver.FileSaver
2020-02-18 10:39:14 +01:00
rawtaz
f2bf06a419
Merge pull request #2540 from dp-github/issue-2531
...
Count hard-linked files correctly in stats command
2020-02-15 21:06:46 +01:00
David Potter
71900248ab
Correct #2531 (`stats` restore-size calculates hard links incorrectly)
2020-02-15 19:34:22 +01:00
David Potter
23055aaadf
Correct #2537 (cmd_stats file counting issue)
2020-02-15 19:34:16 +01:00
rawtaz
8bf6a3af97
Merge pull request #2285 from curiousleo/bugfix/2281-json-output
...
Fix JSON Printf issues with format verbs
2020-02-15 02:00:57 +01:00
curiousleo
27d6e5e186
Add changelog file.
2020-02-15 01:32:43 +01:00
curiousleo
4214b1746e
Ensure Print{,f,ln} use global stdout
2020-02-15 01:32:43 +01:00
curiousleo
f6f240573a
Don't Printf already formatted output
...
Fixes #2281 .
2020-02-15 01:32:43 +01:00
rawtaz
ecdf49679e
Merge pull request #2579 from MichaelEischer/fix-flaky-archiver-test
...
Fix flaky TestArchiverAbortEarlyOnError
2020-02-15 01:29:37 +01:00
Michael Eischer
e1f722d266
archiver: Fix flaky TestArchiverAbortEarlyOnError
...
Each of the random test files was split into the same five blobs. The
test fails once the fifth blob is passed on to `SaveBlob`. That is for
certain interleavings of goroutine execution it would be possible for
the test to trigger the testErr just after storing the first file.
The fixed test uses a different file content for each of the nine files
and fails after writing the fourth blob. The file content is also small
enough to ensure that for each file only a single blob is saved. This
guarantees that the test cannot fail before reading the first four
files. FileReadConcurrency = 2 allows up to two files queued for
processing. Therefore the test can at most open the sixth file before it
has to save the fourth file / blob which triggers the testErr.
2020-02-14 23:16:13 +01:00
rawtaz
2d47381b1d
Merge pull request #2545 from MichaelEischer/fix-racy-backup-json
...
Fix racy json output for backup command
2020-02-13 22:33:17 +01:00
Michael Eischer
294ca55967
Add changelog entry
2020-02-13 21:14:20 +01:00
Michael Eischer
78c518ccac
Print backup summary after status output is shutdown
2020-02-13 21:14:20 +01:00
Michael Eischer
42a3292bcf
Better name for jsonstatus package
...
internal/ui/jsonstatus and termstatus sound similar but are not related
in any way. Instead `internal/ui/backup` and `internal/ui/jsonstatus/status`
are the counterparts. Rename the latter to `internal/ui/json/backup` to
make this clear.
2020-02-13 21:14:20 +01:00
Michael Eischer
ef70a2fcb3
Fix mangled JSON output by backup command
...
jsonstatus wrote the JSON output without synchronization to the
stdio_wrapper which caused mangling between different status lines.
Use the Print and Error methods of termstatus instead which use a
central goroutine to synchronize output.
2020-02-13 21:14:20 +01:00
rawtaz
af20015725
Merge pull request #2287 from benchti/patch-1
...
doc: Correct parameter order in dump example
2020-02-13 01:54:10 +01:00
rawtaz
680a14afa1
Merge pull request #2530 from restic/fix-sftp-mkdirall
...
sftp: Use MkdirAll provided by the client
2020-02-13 01:12:59 +01:00
rawtaz
bf199e5ca7
Minor speling corection.
2020-02-13 00:47:09 +01:00
rawtaz
ae127a412d
Merge pull request #2337 from rfjakob/favicon
...
docs: switch to gopher favicon
2020-02-13 00:14:33 +01:00
Jakob Unterwurzacher
5ae7e6f945
docs: switch to gopher favicon
...
The favicon on restic.readthedocs.io still contained the old
superman-style logo.
This changes it to the 32x32 gopher icon also used on https://forum.restic.net/ ,
just converted to .ico using Gimp.
Tested by building the documentation and opening index.html in Chrome.
The new favicon looks fine.
2020-02-13 00:09:37 +01:00
rawtaz
d8da9c4401
Merge pull request #2577 from alrs/fix-internal-errs
...
internal: Fix code and test dropped errors
2020-02-12 23:41:54 +01:00
rawtaz
daf3bfc882
Merge pull request #2461 from rigtorp/man-exit-status
...
Add documentation on exit status codes to man pages
2020-02-12 23:17:49 +01:00
Erik Rigtorp
94f4f13388
Add documentation on exit status codes to man pages
...
This is step one to start defining useful exit codes for all the commands.
2020-02-12 23:09:26 +01:00
rawtaz
4615bdfd70
Merge pull request #2576 from restic/update-chunker
...
Update the chunker
2020-02-12 22:41:36 +01:00
rawtaz
299f5971f2
Merge pull request #2560 from brualan/master
...
Two small improvements to code quality
2020-02-12 22:40:20 +01:00
rawtaz
7a1352ae87
Merge pull request #2570 from MichaelEischer/close-file-on-type-change
...
Close file if file type has changed after initial stat
2020-02-12 22:39:22 +01:00
Lars Lehtonen
72734d59b5
internal/archiver: fix dropped error
2020-02-12 13:37:37 -08:00
rawtaz
3679669aae
Merge pull request #2574 from alrs/fix-dropped-cmd-err
...
cmd/restic: fix dropped error
2020-02-12 22:36:45 +01:00
Lars Lehtonen
3ed54e762e
internal/backend/sftp: fix dropped test error
2020-02-12 13:36:21 -08:00
Lars Lehtonen
fea835b4e2
internal/repository: fix dropped test error
2020-02-12 13:33:54 -08:00
Lars Lehtonen
16b321b140
internal/restic: fix dropped test error
2020-02-12 13:32:45 -08:00
Alexander Neumann
b58dfda212
Add entry to changelog
2020-02-12 21:29:26 +01:00
Alexander Neumann
b229aa5cf1
Update the chunker
2020-02-12 21:29:26 +01:00
Alexander Neumann
81c0b031f9
Add link to the forum in 'new issue' dialog
2020-02-12 21:25:48 +01:00
Michael Eischer
760863e7f9
archiver: Fix TestRacyFileSwap on windows
2020-02-11 21:09:47 +01:00
Michael Eischer
a135699397
Close file if file type has changed after initial stat
2020-02-11 21:09:47 +01:00
Lars Lehtonen
94a4d45dfb
cmd/restic: fix a dropped error
2020-02-11 09:19:03 -08:00
rawtaz
4de384087d
Merge pull request #2575 from alrs/fix-travis
...
restic/internal/ui: fix gofmt nit that popped up in Go 1.13
2020-02-11 13:49:13 +01:00
Lars Lehtonen
57815d9cd6
restic/internal/ui: fix gofmt nit that popped up in Go 1.13
2020-02-10 11:06:47 -08:00
rawtaz
644673bcf2
Merge pull request #2573 from dhoffend/fix-processed-bytes
...
fix backup --json total_bytes_processed output
2020-02-10 13:35:55 +01:00
Daniel Hoffend
e7cdf2acbb
fix backup --json total_bytes_processed output
...
Closes #2429
2020-02-09 02:02:45 +01:00
rawtaz
0f22f008ed
Merge pull request #2552 from iambryancs/update-to-go1.13
...
Update Go version to 1.13.6 in Docker build script
2020-02-07 15:24:37 +01:00
rawtaz
c184da21d0
Merge pull request #2568 from flofeld/patch-1
...
corrected option s3.storage-class in documentation
2020-02-06 23:34:36 +01:00
Florian Feldmann
ef5efc6a82
corrected option s3.storage-class
...
Took me some time to figure it out
2020-02-06 22:53:47 +01:00
Alexander Bruyako
688014487b
avoid "index out of range" error
...
in case of len(format) == 0, we will get an
"index out of range" error
Usage of strings.HasSuffix is allowing to avoid that
2020-01-27 19:24:42 +03:00
Alexander Bruyako
38ddfbc4d3
simpler error return
2020-01-27 18:41:46 +03:00
Alexander Bruyako
da48b925ff
remove unnecessary error return
...
I was running "golangci-lint" and found this two warnings
internal/checker/checker.go:135:18: (*Checker).LoadIndex$3 - result 0 (error) is always nil (unparam)
final := func() error {
^
internal/repository/repository.go:457:18: (*Repository).LoadIndex$3 - result 0 (error) is always nil (unparam)
final := func() error {
^
It turns out that these functions are used only in "RunWorkers(...)",
which is used only two times in whole project right after this "final"
functions.
And because these "final" functions always return "nil", I've
descided, that it would be better to remove requriments for "final" func
to return error to avoid magick "return nil" at their end.
2020-01-27 18:28:21 +03:00
iambryancs
4bcd56fbae
Update Go version to 1.13.6 in Docker build script
2020-01-17 02:13:11 +08:00
Alexander Neumann
6e9778ae0e
Add changelog file
2020-01-06 21:46:22 +01:00
Alexander Neumann
63c67be908
Add hint abouth absolute sftp paths for Synology NAS
2020-01-06 21:37:57 +01:00
rawtaz
d70a4a9350
Merge pull request #2373 from vrusinov/issue-2372
...
Ignore username difference in TestMetadataChanged
2020-01-05 21:07:34 +01:00
Alexander Neumann
2cd9c7ef16
sftp: Use MkdirAll provided by the client
...
Closes #2518
2020-01-01 17:26:38 +01:00
Alexander Neumann
6ec5dc8016
Fix template for new version of calens
...
The `wrap` function has been renamed to `wrapIndent`
2020-01-01 11:58:21 +01:00
rawtaz
fe430a680a
Merge pull request #2524 from cure/patch-1
...
Fix typos in documentation of FixTime
2019-12-22 16:36:23 +01:00
Ward Vandewege
69a0d0ee90
fix typos
2019-12-21 21:00:28 -05:00
rawtaz
4557881066
Merge pull request #2423 from streambinder/master
...
sftp: support user@domain parsing as user
2019-12-19 22:39:36 +01:00
streambinder
9b5d069ade
doc: 030: sfpt: hint for domain-confined users
2019-12-19 13:15:37 +01:00
streambinder
c56cbfe95b
changelog: add pull-2423
2019-12-19 13:15:37 +01:00
streambinder
97e5ce4344
internal: backend: sftp: support user@domain parsing as user
2019-12-19 13:15:37 +01:00
rawtaz
72bd467e23
Merge pull request #2271 from lorenzbausch/feature/find_timestamp
...
Display snapshot date when using `restic find`
2019-12-17 10:35:59 +01:00
Lorenz Bausch
d818b7618b
Display respective snapshot date when using `restic find`
2019-12-17 07:51:12 +01:00
Matt Holt
1c3812a6f6
Merge pull request #2342 from rpsene/master
...
Add support for ppc64le
2019-12-16 11:50:52 -07:00
Rafael Peria de Sene
ec91b80f09
Add support for ppc64le
...
This commit adds support for cross-compiling Restic for ppc64le
as part of the release process.
Add a new file: changelog/unreleased/issue-2277
Modifies: helpers/build-release-binaries/main.go
Modifies: run_integration_tests.go
2019-12-15 22:52:13 -03:00
rawtaz
18a336c154
Merge pull request #2510 from rawtaz/kastenhq-changelog-2390
...
Add changelog for #2390 .
2019-12-10 23:31:20 +01:00
Eric Hamilton
d21a13f1ee
Create issue-2390
2019-12-10 23:20:18 +01:00
rawtaz
e0eac30ee5
Merge pull request #2509 from restic/improve-check-docs
...
doc: Improve check documentation
2019-12-10 23:07:23 +01:00
rawtaz
fccb579471
doc: Improve check documentation
...
Make it clearer what the difference is between regular check and check with --read-data.
2019-12-10 21:15:59 +01:00
Alexander Neumann
6c700f95b5
Merge pull request #2358 from filippobottega/master
2019-11-29 20:27:30 +01:00
Filippo Bottega
95d070c147
FAQ: Add hint for antivirus setup
...
Add a tip to configure antivirus to exclude restic process.
2019-11-29 20:26:20 +01:00
rawtaz
da4473aba6
Merge pull request #2391 from kastenhq/refresh-lock-time
...
Update Lock.Time in lock.Refresh()
2019-11-26 21:44:29 +01:00
rawtaz
6e85a58045
Merge pull request #2465 from whs-dot-hk/fix-test-metadata-changed
...
Fix test metadata changed
2019-11-26 21:42:42 +01:00
rawtaz
476e2e8762
Merge pull request #2354 from HugoReeves/master
...
Doc Addition: Wasabi repo setup instructions
2019-11-25 13:56:49 +01:00
Alexander Neumann
246bb46e82
Correct CHANEGLOG.md
2019-11-24 15:18:25 +01:00
Alexander Neumann
fe0361ffcb
Move changelog entry for #2179
...
For background on how this happened see
https://forum.restic.net/t/psa-attention-anyone-not-using-a-recent-master-ie-0-9-5-version-restic-will-not-detect-changed-files-if-mtime-is-reset-by-the-application-eg-ms-excel-and-xls-xlsx-files-on-macos/1772/8?u=fd0
2019-11-24 15:15:19 +01:00
rawtaz
952469473f
Merge pull request #2486 from rawtaz/changelog-gitkeep
...
changelog: Add .gitkeep to persist unreleased/ folder
2019-11-24 14:59:39 +01:00
rawtaz
02108f202e
Merge pull request #2487 from rawtaz/2469-fix-diff-stats
...
diff: Fix wrong bytes reported in diff stats (#2469 )
2019-11-23 12:15:28 +01:00
Alexander Neumann
b02357f542
Fix message
2019-11-23 10:31:40 +01:00
Alexander Neumann
f2aeaef8f1
Improve questions
2019-11-23 10:19:08 +01:00
Alexander Neumann
547702d285
Remove coverage badge
2019-11-23 10:19:08 +01:00
rawtaz
599831f874
Merge pull request #2489 from restic/improve-forget-docs
...
Improve example for forget --keep-daily
2019-11-22 21:20:35 +01:00
Alexander Neumann
14c90d9e85
Improve example for forget --keep-daily
...
Following up on https://github.com/restic/restic/pull/2406
2019-11-22 20:44:50 +01:00
Alexander Neumann
c41bbb3761
Re-formulate the question in the issue template
...
Closes #2410
2019-11-22 20:39:47 +01:00
rawtaz
8a54a0f5ac
Merge pull request #2488 from rawtaz/changelog-template
...
changelog: Fix typo in TEMPLATE
2019-11-22 18:23:02 +01:00
Leo R. Lundgren
c0a5530950
changelog: Fix typo in TEMPLATE.
2019-11-22 17:32:56 +01:00
Leo R. Lundgren
77ef92b95c
diff: Fix wrong bytes reported in diff stats ( #2469 )
2019-11-22 17:26:01 +01:00
Leo R. Lundgren
69f75bbf70
changelog: Add .gitkeep to persist the unreleased/ folder.
2019-11-22 17:14:17 +01:00
Alexander Neumann
30519f01ff
Set development version for 0.9.6
2019-11-22 16:19:03 +01:00
Alexander Neumann
b723ca3de5
Add version for 0.9.6
2019-11-22 16:18:59 +01:00
Alexander Neumann
f5084d70d7
Generate CHANGELOG.md for 0.9.6
2019-11-22 16:18:55 +01:00
Alexander Neumann
29b7b17491
Prepare changelog files for 0.9.6
2019-11-22 16:18:33 +01:00
rawtaz
e14c4b1737
Merge pull request #2484 from restic/add-s3-region
...
s3: Allow specifying region
2019-11-22 15:51:17 +01:00
Alexander Neumann
745d79fe5f
Merge pull request #2483 from restic/update-minio-go
...
Update minio-go
2019-11-22 15:31:12 +01:00
Alexander Neumann
fb95426f64
Rename environment variable to AWS_DEFAULT_REGION
...
This seems to be the correct name, at least the AWS cli uses it:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html
2019-11-22 15:24:42 +01:00
Alexander Neumann
4cadc89ad3
Add documentation and changelog
2019-11-22 15:16:59 +01:00
Alexander Neumann
409909a7f5
Add option description for Region
2019-11-22 15:09:09 +01:00
mdauphin
df500a372d
Add AWS_REGION env var to specify s3 region
2019-11-22 15:04:48 +01:00
Alexander Neumann
a444731dc0
Update vendored dependencies
2019-11-22 14:57:56 +01:00
Alexander Neumann
a6e8af7e0f
Update minio-go
2019-11-22 14:50:46 +01:00
Alexander Neumann
aa5af8af0e
Merge pull request #2478 from restic/update-ci
...
Update Go version for Appveyor/Travis
2019-11-22 14:36:38 +01:00
Alexander Neumann
4e3353109d
Reset changes by 'go get'
2019-11-22 13:17:52 +01:00
Alexander Neumann
02c8d38095
Fix error check for findGoFiles
2019-11-22 12:20:27 +01:00
Alexander Neumann
fd6211653c
More debug for findImports
2019-11-22 12:16:46 +01:00
rawtaz
3d4f2dd6b4
Merge pull request #2368 from rlue/doc/backblaze
...
Clarify docs for Backblaze B2 support
2019-11-22 11:10:22 +01:00
Ryan Lue
c1ddc0c18b
Clarify docs for Backblaze B2 support
2019-11-22 18:07:29 +08:00
rawtaz
c95f032a9c
Merge pull request #2310 from vincentbernat/fix/doc-contributing
...
Update documentation for contributing
2019-11-21 20:30:49 +01:00
Vincent Bernat
3087776135
Update documentation for contributing
...
- No need to checkout into `GOPATH` anymore
- `CHANGELOG.md` shouldn't be updated directly
2019-11-21 20:27:48 +01:00
rawtaz
b6f01ffbe6
Merge pull request #2394 from bugvillage/doc-backup-passwd
...
add description env var RESTIC_PASSWORD_COMMAND
2019-11-20 23:26:55 +01:00
rawtaz
41fe9318b1
Merge pull request #2425 from thiell/restic_cache_dir_env
...
Add support for $RESTIC_CACHE_DIR
2019-11-20 21:18:57 +01:00
Alexander Neumann
8387d18d4d
Update Go version used for CI tests
2019-11-20 21:13:56 +01:00
rawtaz
929d2b8df3
Merge pull request #2479 from rawtaz/sing-pulitzer-group
...
Accept both singular and plural for --group-by (#2330 )
2019-11-20 21:03:05 +01:00
Leo R. Lundgren
4f0682d730
Correct name of changelog file for #2321
2019-11-20 21:01:10 +01:00
Alexander Neumann
967d1bbf0c
Run 'go mod tidy'
2019-11-20 20:54:43 +01:00
Alexander Neumann
2f80b37b93
Update go-autorest so restic builds with Go 1.13
2019-11-20 20:53:57 +01:00
Leo R. Lundgren
4d2aa18273
Accept both singular and plural for --group-by ( #2330 )
2019-11-20 19:25:48 +01:00
Alexander Neumann
6b1e5d4e18
Update Go version for Appveyor/Travis
2019-11-20 18:32:58 +01:00
rawtaz
26d1f9f4ba
Merge pull request #2307 from gary-kim/fix-2306-password-retries
...
Allow multiple retries for interactive password input
2019-11-20 18:30:20 +01:00
rawtaz
6a89c0f0ef
Merge pull request #2476 from rawtaz/update-doc
...
doc: Simplify wording for check's --read-data-subset parameter.
2019-11-19 03:38:37 +01:00
rawtaz
b87230b93d
Merge pull request #2456 from jenting/installation-opensuse
...
Docs: add installation guide for openSUSE
2019-11-19 02:15:19 +01:00
JenTing Hsiao
6f2b8d622a
Add installation guide for openSUSE
...
Signed-off-by: JenTing Hsiao <jenting.hsiao@suse.com>
2019-11-19 09:08:18 +08:00
Leo R. Lundgren
90440212f2
doc: Simplify wording for check's --read-data-subset parameter.
2019-11-19 00:43:41 +01:00
rawtaz
3a5c9aadad
Merge pull request #2444 from chanibal/patch-2
...
Docs: fixed escaping of argument in documentation
2019-11-18 21:45:00 +01:00
rawtaz
a78142c1bb
Merge pull request #2450 from hallamjeff/patch-1
...
Update 080_examples.rst
2019-11-18 21:43:19 +01:00
rawtaz
07045c7e23
Merge pull request #2442 from lbausch/patch-1
...
Docs: Highlight option correctly
2019-11-18 21:39:07 +01:00
rawtaz
0a5d42db3f
Merge pull request #2463 from whs-dot-hk/fix-ci-rclone
...
Update rclone repo
2019-11-18 21:37:02 +01:00
Jeff Hallam
67d99b8cfb
Update 080_examples.rst
...
added backticks around the -o and --option text.
2019-11-18 16:36:20 -04:00
Alexander Neumann
1a0c0dc277
Remove codecov.io
...
While it was a nice idea, some tests like the backend integration tests
required credentials which were not available to third-party PRs. This
lead to a lot of uncovered code which confused people. So let's disable
codecov.io for now.
2019-11-18 21:21:40 +01:00
rawtaz
e86d9307d0
Merge pull request #2472 from rawtaz/update-backup-doc
...
doc: Improve exclude/include patterns info
2019-11-18 21:04:18 +01:00
rawtaz
923e681af3
Merge pull request #2471 from 8176135/patch-1
...
Small fix to the forget --tag command documentation
2019-11-18 20:40:24 +01:00
Leo R. Lundgren
37770b1d82
doc: Improve include patterns info
...
Clarifies and adds information on how to specify include patterns.
2019-11-18 02:23:01 +01:00
rawtaz
02fea4f76a
doc: Improve exclude patterns info
...
Improves grammar, clarifies and adds information on how to specify exclude patterns.
2019-11-18 01:59:50 +01:00
whs
7cacba0394
Assume WithAtime default to false
2019-11-06 16:38:46 +08:00
whs
e6db3596f1
Update rclone repo
2019-11-06 11:59:48 +08:00
8176135
3acc7af310
Small fix to the forget --tag command
...
As described in #2460
2019-11-03 11:48:56 +13:00
Jeff Hallam
5c4653f427
Update 080_examples.rst
...
Added an explanation of how to specify the AWS S3 storage classes and what the default value is.
2019-10-21 19:24:50 -03:00
Krzysztof Bociurko
f7317a9287
Docs: fixed escaping of argument in documentation
...
Issue was visible as `foo`` in https://restic.readthedocs.io/en/latest/040_backup.html
2019-10-16 19:03:40 +02:00
lbausch
30db8057e4
Docs: Highlight option correctly
2019-10-13 21:19:03 +02:00
Stephane Thiell
0e897ef7b8
Add support for $RESTIC_CACHE_DIR
...
Add support for restic-specific $RESTIC_CACHE_DIR environment variable
to override the cache directory like --cache-dir would have.
2019-09-26 15:59:56 -07:00
bugvillage
b3e727f40d
fixing typo/align around RESTIC_PASSWORD_COMMAND
2019-09-07 15:30:23 +02:00
bugvillage
17feccd998
add description env var RESTIC_PASSWORD_COMMAND
...
The environment variable RESTIC_PASSWORD_COMMAND works but has
not been documented yet. e.g. it could contain a command that
would fetch the password from a local user keyring
enhances: https://github.com/restic/restic/pull/2094
2019-09-07 14:00:53 +02:00
Eric Hamilton
1596d06f8e
Update Lock.Time in lock.Refresh()
2019-09-04 11:38:35 -07:00
Vladimir Rusinov
db20c0b8d0
Ignore username difference in TestMetadataChanged.
...
In some (rare) cases "fake" UID 51234 may exist in a system running a
test. When this is the case, `cmp.Equal(want, node3)` will fail based on
difference between empty string and an actual username present in a
system.
Fixes github issue #2372
2019-08-13 22:25:00 +01:00
Alexander Neumann
604b18aa74
build: Fix building in ~/go
...
Before, build.go only unset GO111MODULE and GOPATH, so the Go compiler
did not see either and worked in Module mode. But if the code is checked
out below ~/go (the default GOPATH), it will detect that the source is
within GOPATH and switch to non-Module mode. Now we're setting
GO111MODULE to "on" explicitly.
2019-07-29 09:22:22 +02:00
Alexander Neumann
01c51b3449
Merge pull request #2321 from restic/fix-index-error-handling
...
Check errors returned by LoadIndex()
2019-07-27 09:37:18 +02:00
Alexander Neumann
de8cf5e345
Merge pull request #2324 from jkahrs/fix/change-cleanup-cache-message
...
updade message that is shown when a cache cleanup is advised
2019-07-27 09:37:10 +02:00
Alexander Neumann
cfa2ac69e0
Merge pull request #2322 from brualan/master
...
minor refactoring
2019-07-27 09:37:06 +02:00
Hugo Reeves
3ca306d69a
Doc Addition: Added Wasabi repo setup
2019-07-26 22:00:48 +12:00
Alexander Neumann
1e9eefa066
Debian stable has restic 0.9.4 ( #2333 )
...
Debian stable has restic 0.9.4
2019-07-11 08:17:40 +02:00
Michael M. Chang
e9af012229
Debian has updated restic
...
Debian now has restic 0.9.4+ds-2+b1 in stable (buster)
2019-07-10 11:53:17 -04:00
Jean Kahrs
8066e93f47
updade message that is shown when a cache cleanup is advised
2019-07-02 10:49:49 +02:00
Alexandr Bruyako
e19622e4b1
start using fileMode const
2019-07-01 00:26:00 +03:00
Alexandr Bruyako
38ea7ed4f6
remove unused code
2019-07-01 00:24:45 +03:00
Alexandr Bruyako
76d1866444
avoiding unnecessary type conversions
2019-06-30 23:58:00 +03:00
Alexandr Bruyako
8b22fe29cf
improved slice copying
2019-06-30 23:56:36 +03:00
Alexandr Bruyako
02014be76c
simplified prefix removal, removed unnecessary if-else statements
2019-06-30 23:34:47 +03:00
Alexandr Bruyako
16eeed2ad5
simplified string sorting by using a more suitable function
2019-06-30 23:20:32 +03:00
Alexander Neumann
3f94f63967
Add entry to changelog
2019-06-30 21:38:10 +02:00
Alexander Neumann
88716794e3
Check errors returned by LoadIndex()
...
Bug was reported in the forum here: https://forum.restic.net/t/check-rebuild-index-prune/1848/13
2019-06-30 21:34:53 +02:00
Fred Akalin
3ca424050f
Add upper bound for t in --read-data-subset=n/t ( #2304 )
...
* Add upper bound for t in --read-data-subset=n/t
* Add changelog entry
2019-06-29 14:34:53 -04:00
Gary Kim
fea2464d4d
Allow multiple retries for interactive password input
...
Restic used to quit if the repository password was typed incorrectly once.
Restic will now ask the user again for the repository password if typed incorrectly.
The user will now get three tries to input the correct password before restic quits.
2019-06-13 20:11:02 +08:00
Ben
5d272e5c08
Dump : parameter order in sample
...
Usage:
restic dump [flags] snapshotID file
2019-05-28 18:20:52 +02:00
Alexander Neumann
5bd5db4294
Merge pull request #2206 from garrmcnu/fs-stdin-directory
...
fs: Handle absolute pathname for --stdin-filename
2019-05-08 16:05:37 +02:00
Garry McNulty
4429a66b5f
backup: Convert relative pathname for --stdin-filename to absolute ( #2063 )
2019-05-08 15:21:27 +02:00
Garry McNulty
8066195e6e
fs: Handle absolute pathname for --stdin-filename
...
Return valid directory info from Lstat() for parent directories of the
specified filename. Previously only "/" and "." were valid directories.
Also set directory mode as this is checked by archiver.
Closes #2063
2019-05-08 15:21:27 +02:00
Garry McNulty
f7f14cf8c9
fs: Add file info base name check in reader tests ( #2063 )
2019-05-08 15:21:27 +02:00
Garry McNulty
5096f3b491
fs: Update directory check in reader tests ( #2063 )
2019-05-08 15:21:27 +02:00
Alexander Neumann
cf3fc2a5b1
Merge pull request #2266 from restic/fix-windows-tests
...
Fix tests on Windows
2019-05-05 21:34:04 +02:00
Alexander Neumann
920d458a4a
archiver: Use untyped constants for testing FileInfo
2019-05-05 14:57:38 +02:00
Alexander Neumann
b016dc2ff0
archiver/Windows: Skip test new-content-same-filestamp
2019-05-05 14:02:11 +02:00
Alexander Neumann
355db0bc29
windows: Use LastWriteTime for ctime and mtime
...
Windows does not have a concept of a `change time` in the sense as Unix
has it: the field `CreationTime` of the `Win32FileAttributeData` struct
is not updated when attributes or content is changed. So from now on
we're using the `LastWriteTime` as the `change time` on Windows.
2019-05-05 14:02:11 +02:00
Alexander Neumann
6e2fe73189
archiver: Move tests back into the same file
...
Move all Archiver tests back into `archiver_test.go` and add some tiny
helpers to mock what `lstat` returns (for Windows and Unix separately).
2019-05-05 14:02:11 +02:00
Alexander Neumann
303a5dab6a
archiver: Clarify value in test struct
...
Since I could not remember what the value for `Check` means this commit
renames it to `SameFile`: when set to true, the test should make sure
that `FileChanged` should return false (=file is unmodified).
2019-05-05 12:57:00 +02:00
Alexander Neumann
7dcd2968b6
Merge pull request #2261 from lorenzbausch/patch-1
...
Fix typo
2019-05-04 10:23:49 +02:00
Lorenz Bausch
298f490195
Fix typo
2019-05-02 11:51:35 +02:00
Alexander Neumann
37cb82b28b
Merge pull request #2257 from gliptak/windows1
...
Don't run TestMetadataChanged test on Windows
2019-04-28 08:39:00 +02:00
Gábor Lipták
bce6438d22
Don't run TestMetadataChanged test on Windows
...
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2019-04-27 21:23:47 -04:00
Alexander Neumann
919dd2ac84
Merge pull request #2252 from restic/fix-2249
...
Read fresh metadata for unmodified files
2019-04-25 09:15:50 +02:00
Alexander Neumann
870bc5108e
Merge pull request #2253 from restic/fix-2174
...
Make sure timestamps are valid
2019-04-25 09:14:43 +02:00
Alexander Neumann
418296c5c9
Add hint for "closes" wording to PR template
2019-04-25 09:00:15 +02:00
Alexander Neumann
a6481b3707
Merge pull request #2212 from cbane/check-ctime
...
Examine file ctime when checking if files have changed.
2019-04-25 08:58:27 +02:00
Courtney Bane
00b527fb09
Update changelog text, and add pull request link.
2019-04-24 20:54:15 -05:00
Courtney Bane
0ebfc55ee3
Use existing setTimestamp function for ctime test and improve error checking.
2019-04-24 20:53:08 -05:00
Courtney Bane
35b7607802
Don't check ctime when ignoring inode.
2019-04-24 20:53:08 -05:00
Alexander Neumann
fad9f65c65
Merge pull request #2251 from restic/update-deps
...
Update dependencies
2019-04-24 16:42:57 +02:00
Alexander Neumann
939f3e972c
node: Make sure year of all timestamps is valid
...
Sometimes restic gets bogus timestamps which cannot be converted to
JSON, because the stdlib JSON encoder returns an error if the year is
not within [0, 9999]. We now make sure that we at least record _some_
timestamp and cap the year either to 0000 or 9999. Before, restic would
refuse to save the file at all, so this improves the status quo.
This fixes #2174 and #1173
2019-04-24 16:39:36 +02:00
Alexander Neumann
ca8c3b4fd5
Update dependencies
2019-04-24 15:17:48 +02:00
Alexander Neumann
4f45b14f25
Add changelog file
2019-04-24 15:17:25 +02:00
Alexander Neumann
389067fb8b
Only use list of blobs for old node
...
Closes #2249
2019-04-24 15:07:26 +02:00
Alexander Neumann
4b0ca9ddab
Add test for #2249
2019-04-24 15:07:23 +02:00
Courtney Bane
b8c2544dcb
Examine file ctime when checking if files have changed.
2019-04-23 21:54:35 -05:00
Alexander Neumann
c7762453cf
Set development version for 0.9.5
2019-04-23 13:19:43 +02:00
Alexander Neumann
303210aa08
Add version for 0.9.5
2019-04-23 13:19:39 +02:00
Alexander Neumann
c029881379
Update manpages and auto-completion
2019-04-23 13:19:39 +02:00
Alexander Neumann
6e89963c21
Generate CHANGELOG.md for 0.9.5
2019-04-23 13:19:36 +02:00
Alexander Neumann
1ac560181b
Prepare changelog for 0.9.5
2019-04-23 13:19:23 +02:00
Alexander Neumann
18ec27a0da
Merge pull request #2243 from treymo/ls-length-2104
...
backend find: Check prefix length < snapshot IDs when searching
2019-04-23 09:41:19 +02:00
Alexander Neumann
b40dea29ad
Merge pull request #2247 from tbm/docs
...
Minor docs fixes
2019-04-23 09:17:22 +02:00
Martin Michlmayr
0561155963
Split paragraph from list in restic stats --help
...
The help text for `restic stats` lists a number of modes in a list.
Make sure the "more info" text is a separate paragraph rather than
being part of the list.
2019-04-23 10:36:33 +07:00
Martin Michlmayr
1aafc17212
Avoid error "undefined control sequence" in LaTeX
...
Converting the changelog to PDF using pandoc leads to:
! Undefined control sequence.
l.1497 ...mple, by creating a file named ``..\test
This is because \t is interpreted as a control sequence. Use ``
instead of "" to work around this.
2019-04-23 10:36:33 +07:00
Martin Michlmayr
f11789c437
Add missing opening parenthesis
2019-04-23 10:36:26 +07:00
Alexander Neumann
8cab0c121d
Merge pull request #2221 from classmarkets/2203-password-from-stdin
...
Fix reading password from stdin
2019-04-22 20:59:59 +02:00
Alexander Neumann
5979414bcd
Merge pull request #2124 from Kidswiss/tar
...
Ability to dump folders to tar via stdout
2019-04-22 20:58:59 +02:00
Simon Beck
cc8b690b52
Restore whole folder to sdtout as tar
...
With this change it is possible to dump a folder to stdout as a tar. The
It can be used just like the normal dump command:
`./restic dump fa97e6e1 "/data/test/" > test.tar`
Where `/data/test/` is a a folder instead of a file.
2019-04-22 20:44:33 +02:00
Alexander Neumann
a164dc9391
Merge pull request #2087 from ArcticXWolf/add_group_by_option_for_snapshots
...
Add GroupBy option to snapshots command
2019-04-22 20:27:24 +02:00
Jan Niklas Richter
9a26be4e5b
Add documentation for --group-by flag for snapshots command
2019-04-22 14:57:53 +02:00
Jan Niklas Richter
733519d895
Refactor duplicate code for grouping snapshots
...
This commit is a followup to the addition of the --group-by flag for the
snapshots command. Adding the grouping code there introduced duplicated
code (the forget command also does grouping). This commit refactors
boths sides to only use shared code.
2019-04-22 14:57:53 +02:00
Jan Niklas Richter
3d5a0c799b
Add changelog of group-by option for snapshots command
2019-04-22 14:37:58 +02:00
Jan Niklas Richter
c4475ac58f
Move snapshot grouping code into own function to deduplicate code
...
This commit moves the code which is used to group snapshots in the
snapshots command into an own function to deduplicate code shared by the
snapshots command and forget command.
2019-04-22 14:37:58 +02:00
Jan Niklas Richter
c9fd9b5275
Fix json tags for grouped snapshot output
...
This commit will add json tags to the structs for json output, so all
json variables of the snapshot command output are lowercase and
snake-case.
Furthermore it adds some internal code changes based on the feedback in
the pull request #2087 .
2019-04-22 14:37:58 +02:00
Jan Niklas Richter
cadcab5a19
Add GroupBy option to snapshots command
...
This commit adds a --group-by option to the snapshots command, which
behaves similar to the --group-by option of forget. Valid option values
are "host, paths, tags". If this option is given, the output of
snapshots will be divided into multiple tables, according to the value
given (i.e. "host" will create a table of snapshots for each host, that
has a snapshot in the list). Also the JSON output will be grouped.
The default behavior (when --group-by is not given) has not changed.
More to this discussion can be found in issue #2037 .
2019-04-22 14:37:58 +02:00
Trey Moore
5ac9c1157a
Prevent "slice bounds out of range" error if prefix is longer than snapshot IDs.
...
This includes tests as well as other tests for the backend_find.Find function. Issue #2104 .
2019-04-16 20:26:20 -07:00
Peter Schultz
5715517e29
Fix reading password from stdin
...
Reading the password from non-terminal stdin used io.ReadFull with a
byte slice of length 1000.
We are now using a Scanner to read one line of input, independent of its
length.
Additionally, if stdin is not a terminal, the password is read only
once instead of twice (in an effort to detect typos).
Fixes #2203
Signed-off-by: Peter Schultz <peter.schultz@classmarkets.com>
2019-04-15 10:05:49 +02:00
Alexander Neumann
ecc2458de8
Merge pull request #2230 from cdhowie/issue-2229
...
Extend find --show-pack-id to work with --tree
2019-04-14 19:38:38 +02:00
Alexander Neumann
2c6ba5d9ac
Merge pull request #2231 from cdhowie/issue-2224
...
find command: don't abort on tree load errors
2019-04-14 19:31:06 +02:00
Alexander Neumann
0cc3647e51
Remove changelog entry, not relevant for end-users
2019-04-14 19:29:12 +02:00
Alexander Neumann
6b700d02f5
Merge pull request #2217 from restic/improve-memory-usage
...
WIP: improve memory usage
2019-04-13 15:07:07 +02:00
Alexander Neumann
2b09a10234
Merge pull request #2218 from kcthrn/patch-1
...
Fix minor error in doc for forgetting snapshots
2019-04-13 14:55:40 +02:00
Alexander Neumann
1c87d01bad
Merge pull request #2220 from BenoitKnecht/s3-storage-class
...
Add config option to set S3 storage class
2019-04-13 14:54:32 +02:00
Alexander Neumann
78a3ffcfb9
Merge pull request #2185 from d3zd3z/json-forget
...
Json forget
2019-04-13 14:23:32 +02:00
Alexander Neumann
4d77c0c21c
Merge pull request #2228 from cstegm/patch-1
...
Update 030_preparing_a_new_repo.rst
2019-04-13 14:20:01 +02:00
Alexander Neumann
fb064afa34
Merge pull request #2232 from jacksgt/document_cachedir-tag
...
Mention Cache Directoy Tagging Standard in man page and help text
2019-04-13 14:17:38 +02:00
Alexander Neumann
7304738872
check: Reduce default parallelism from 40 to 5
2019-04-13 13:38:39 +02:00
Alexander Neumann
66efa425bf
Reuse buffer in worker functions
2019-04-13 13:38:39 +02:00
Alexander Neumann
d51e9d1b98
Add []byte to repo.LoadAndDecrypt and utils.LoadAll
...
This commit changes the signatures for repository.LoadAndDecrypt and
utils.LoadAll to allow passing in a []byte as the buffer to use. This
buffer is enlarged as needed, and returned back to the caller for
further use.
In later commits, this allows reducing allocations by reusing a buffer
for multiple calls, e.g. in a worker function.
2019-04-13 13:38:39 +02:00
Alexander Neumann
e046428c94
Replace FilesInParallel with an errgroup.Group
2019-04-13 13:38:39 +02:00
Alexander Neumann
75906edef5
mock: Remove unused repository type
2019-04-13 13:38:39 +02:00
Alexander Neumann
203d775190
restic: Make JSON unmarshal for ID more efficient
...
This commit reduces several allocations in UnmarshalJSON() by decoding
the hex string directly in a single step.
2019-04-13 13:38:39 +02:00
Jack Henschel
ecd7ee85e8
Mention Cache Directoy Tagging Standard in man page and help text
...
This patch makes it more explicit what is meant by the CACHEDIR.TAG file.
It not only has to have this particular name, but also a specific content
(described at http://bford.info/cachedir/spec.html ), which is not immediately
obvious to the user.
2019-04-02 15:53:07 +02:00
Alexander Neumann
2022355800
doc: Fix versions in self-update example
2019-03-30 09:44:59 +01:00
Chris Howie
36f22a0feb
find command: don't abort on tree load errors
2019-03-29 22:58:12 -04:00
Chris Howie
f58a44b911
Extend find --show-pack-id to work with --tree
2019-03-29 22:56:36 -04:00
cstegm
fe886a6439
Update 030_preparing_a_new_repo.rst
...
Helps users having this issue: https://github.com/restic/restic/issues/2216
2019-03-29 18:24:17 +01:00
Benoît Knecht
be23313072
changelog: Add entry for s3.storage-class option
2019-03-26 16:39:27 +01:00
Benoît Knecht
3c112d9cae
s3: Add config option to set storage class
...
The `s3.storage-class` option can be passed to restic (using `-o`) to
specify the storage class to be used for S3 objects created by restic.
The storage class is passed as-is to S3, so it needs to be understood by
the API. On AWS, it can be one of `STANDARD`, `STANDARD_IA`,
`ONEZONE_IA`, `INTELLIGENT_TIERING` and `REDUCED_REDUNDANCY`. If
unspecified, the default storage class is used (`STANDARD` on AWS).
You can mix storage classes in the same bucket, and the setting isn't
stored in the restic repository, so be sure to specify it with each
command that writes to S3.
Closes #706
2019-03-26 16:37:07 +01:00
kcthrn
2970e38d92
Fix minor error in doc for forgetting snapshots
2019-03-24 17:59:19 -04:00
Alexander Neumann
870e7583a1
Merge pull request #2210 from restic/update-deps
...
Update dependencies
2019-03-17 13:38:34 +01:00
Alexander Neumann
db1c835c37
Update dependencies
2019-03-17 12:13:53 +01:00
Alexander Neumann
190bed9908
Merge pull request #2208 from restic/update-go
...
Update Go for CI tests
2019-03-17 11:36:06 +01:00
Alexander Neumann
85f4c826db
Update Go for CI tests
2019-03-17 11:06:16 +01:00
Alexander Neumann
5da4b0fc7d
Merge pull request #2209 from restic/fix-freebsd
...
Fix build on FreeBSD for Go < 1.12
2019-03-17 11:05:38 +01:00
Alexander Neumann
c1058005c3
Fix build on FreeBSD for Go < 1.12
2019-03-16 20:52:37 +01:00
Alexander Neumann
ca73808649
Merge pull request #2197 from dmgk/master
...
Fix build on FreeBSD with Go 1.12
2019-03-16 20:23:19 +01:00
Alexander Neumann
f2ea91df38
Merge pull request #2189 from ifedorenko/filewrite-race
...
Fix rare 'file already closed' during restore
2019-03-16 19:53:18 +01:00
Alexander Neumann
15cc4d74b2
Run 'go mod tidy' and 'go mod vendor'
2019-03-16 17:01:20 +01:00
Igor Fedorenko
bf9a507148
Fix rare 'file already closed' during restore
...
Fixes #2183
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2019-03-16 17:01:20 +01:00
Alexander Neumann
65b476ead9
Fix gofmt
2019-03-16 13:29:05 +01:00
Alexander Neumann
aaa1cc2c26
Merge pull request 2193 from restic/allow-empty-rclone-args
...
rclone: Rework backend option parsing
2019-03-16 12:17:38 +01:00
Alexander Neumann
95434cff16
Merge pull request 2156 from kayrus/swift-app-cred
...
Swift: introduce application credential auth support
2019-03-16 12:17:17 +01:00
Alexander Neumann
1b94ae1c00
Merge pull request 2205 from heikobornholdt/ignore-inode
...
Add --ignore-inode option to backup cmd
2019-03-16 12:16:31 +01:00
Heiko Bornholdt
d138b38f28
Add pull-2205 to changelog
2019-03-10 21:37:01 +01:00
Heiko Bornholdt
db8f5864fc
Add --ignore-inode option to backup cmd
...
revised version of https://github.com/restic/restic/pull/2047
2019-03-10 21:24:29 +01:00
Dmitri Goutnik
1d8b21cdad
Fix build on FreeBSD with Go 1.12
2019-03-06 09:41:49 -05:00
Alexander Neumann
3865b59716
rclone: Rework backend option parsing
...
This change allows passing no arguments to rclone, using `-o
rclone.args=""`. It is helpful when running rclone remotely via SSH
using a key with a forced command (via `command=` in `authorized_keys`).
2019-03-02 10:36:42 +01:00
Alexander Neumann
7b8d1dc040
Merge pull request 2187 from restic/fix-2181
...
self-update: Don't cancel download after 30s
2019-02-24 21:21:55 +01:00
David Brown
d19a29f79e
Include reasons in json output of forget
...
This dumps the reasons as well as the list of keeps and removes with the
output from the forget command.
2019-02-23 09:38:33 -07:00
David Brown
449c049ce9
Add changelog for #2184
2019-02-23 09:37:38 -07:00
Alexander Neumann
9f436d80e1
Add changelog file
2019-02-23 11:16:08 +01:00
Alexander Neumann
e277a92a2f
self-update: Don't cancel download after 30 seconds
2019-02-23 11:15:18 +01:00
David Brown
d9e22c2df1
Add test for `--json` support for `forget` command
...
This adds a test of the json output of the forget command, by running it
once, asking it to keep one snapshot, and verifying that the output has
the right number of snapshots listed in the Keep and Remove fields of
the result.
2019-02-22 22:00:20 -07:00
David Brown
4b0fb5af36
Add `--json` support to `forget` command
...
Fixes #2184
2019-02-22 22:00:15 -07:00
Alexander Neumann
7519c73987
Clarify documentation on --exclude-if-present
2019-02-22 15:22:29 +01:00
Alexander Neumann
45a48eb4a8
Merge pull request 2171 from tbm/typos
...
Fix typos
2019-02-21 19:46:10 +01:00
Martin Michlmayr
a2f30cde4c
Fix typos
2019-02-17 15:31:07 +00:00
kayrus
6ebcfe7c18
Swift: introduce application credential auth support
2019-02-14 14:19:05 +01:00
Alexander Neumann
0022926eba
Merge pull request 2167 from moritzdietz/master
...
Update dead link in docs
2019-02-11 21:10:44 +01:00
Alexander Neumann
3e3a0220ec
Merge pull request 2151 from restic/update-deps
...
Update dependencies
2019-02-11 20:27:17 +01:00
Moritz Dietz
c125fb763d
Update dead link
2019-02-11 20:18:43 +01:00
Alexander Neumann
b9f0f031b6
Update dependencies
...
Closes #2129
2019-02-10 13:24:37 +01:00
Alexander Neumann
aa7043151a
Merge pull request 2147 from kilburn/macports
...
Document installation with MacPorts
2019-02-10 12:50:52 +01:00
Alexander Neumann
ebf22a35f4
Merge pull request 2154 from ciderpunx/patch-1
...
Remove stray "in" from b2 instructions
2019-02-10 12:50:11 +01:00
Alexander Neumann
3f069ac404
Merge pull request 2153 from dmd/umount-message
...
more accurate umount help message, for #815
2019-02-10 12:43:48 +01:00
Alexander Neumann
56e5467096
Merge pull request 2032 from j6s/feature.case-insensitive-exclude
...
Add options for case insensitive includes & excludes
2019-02-10 12:40:33 +01:00
Alexander Neumann
5ee932a124
Add changelog for #2139
2019-02-10 12:35:41 +01:00
Alexander Neumann
fed25714a4
Merge pull request 2139 from restic/fail-zero-bytes-stdin
...
Return error when reading zero byte from stdin
2019-02-10 12:34:05 +01:00
Alexander Neumann
8906d85ab8
Merge pull request 2138 from restic/fix-2136
...
Use processed bytes for summary
2019-02-10 12:32:50 +01:00
Alexander Neumann
97aafc1eec
Add changelog file for #1944
2019-02-10 12:31:18 +01:00
Alexander Neumann
6a5c9f57c2
Merge pull request 1944 from mholt/jsonprogress
...
backup: Support --json flag by streaming JSON to report progress
2019-02-10 12:29:50 +01:00
Matthew Holt
6cf13483b5
Add snapshot ID to summary output
2019-02-10 12:09:09 +01:00
Matthew Holt
f645306a18
Initial implementation of JSON progress reporter
2019-02-10 12:09:09 +01:00
ciderpunx
186e10e0cb
Remove stray "in" from b2 instructions
2019-01-29 12:39:46 +00:00
Daniel M. Drucker
29a5bd5b30
more accurate umount help message, for #815
2019-01-28 16:53:35 -05:00
Marc Pujol
06a01bc016
Document installation with MacPorts
2019-01-21 08:13:53 +01:00
Johannes Hertenstein
cdc287a7f6
Store reference to reject function for insensitive pattern rejection
2019-01-19 11:08:13 +00:00
Johannes Hertenstein
deedc38129
Only convert iexclude & iinclude casing once
2019-01-19 11:04:53 +00:00
Alexander Neumann
1107eef215
Merge pull request 2142 from restic/fix-2140
...
Print message in verbose mode only
2019-01-12 13:30:20 +01:00
Alexander Neumann
60c7020bcb
Print message in verbose mode only
...
Closes #2140
2019-01-10 21:28:22 +01:00
Alexander Neumann
b96ef48562
Set development version for 0.9.4
2019-01-06 21:40:32 +01:00
Alexander Neumann
cd9b2295f1
Add version for 0.9.4
2019-01-06 21:40:32 +01:00
Alexander Neumann
a439cdeb05
Update manpages and auto-completion
2019-01-06 21:40:32 +01:00
Alexander Neumann
827f6d7b24
Generate CHANGELOG.md for 0.9.4
2019-01-06 21:40:31 +01:00
Alexander Neumann
77ab10d401
Create changelog dir for 0.9.4
2019-01-06 21:39:56 +01:00
Alexander Neumann
3b0ad2e368
release: Pull latest alpine image
2019-01-06 21:39:56 +01:00
Alexander Neumann
2996c110f1
Merge pull request 2017 from kylewlacy/fuse_default_permissions_option
...
mount: Add default-permissions flag to set FUSE option
2019-01-06 21:19:27 +01:00
Alexander Neumann
4609b5c24d
Merge pull request 2094 from juergenhoetzel/password-command
...
Add support for reading password from external command
2019-01-06 21:14:52 +01:00
Alexander Neumann
830511460a
mount: Turn on DefaultPermissions for --allow-other`
...
This commit changes the logic slightly: checking the permissions in the
fuse mount when nobody else besides the current user can access the fuse
mount does not sense. The current user has access to the repo files in
addition to the password, so they can access all data regardless of what
the fuse mount does.
Enabling `--allow-root` allows the root user to access the files in the
fuse mount, for this user no permission checks will be done anyway.
The code now enables `DefaultPermissions` automatically when
`--allow-other` is set, it can be disabled with
`--no-default-permissions` to restore the old behavior.
2019-01-06 20:55:49 +01:00
Alexander Neumann
0dc3648416
Add documentation for `--password-command`
2019-01-06 20:41:38 +01:00
Alexander Neumann
d71dba3788
Merge pull request 2130 from andreaso/docs-https-links
...
Prefer https:// links in the documentation
2019-01-06 20:24:42 +01:00
Alexander Neumann
e482633943
Merge pull request 2137 from kevinawoo/patch-1
...
doc: Add more clarity to B2 application key
2019-01-06 20:24:00 +01:00
Alexander Neumann
900621051a
Merge pull request 2120 from turnkey-commerce/fix-doc-typo
...
Fix a few typos in the FAQ doc.
2019-01-06 15:08:13 +01:00
Alexander Neumann
1f246c5309
Merge pull request 2090 from plumbeo/within-hours
...
Increase the granularity of the "keep within" snapshot retention policy
2019-01-06 15:04:55 +01:00
Alexander Neumann
e40805b002
Merge pull request 2103 from cdhowie/master
...
--one-file-system unsupported on Windows
2019-01-06 14:59:46 +01:00
Alexander Neumann
6f69ae1b8d
Add docs, changelog file
2019-01-06 14:52:20 +01:00
Alexander Neumann
c4fbf2c779
Return error when reading zero byte from stdin
...
This commit changes the internal file system implementation for reading
data from stdin, it now returns an error when no bytes could be read. I
think it's worth failing in this case, the user instructed restic to
read some data from stdin, and no data was read at all. Maybe it was in
a pipe and some earlier stage failed.
See #2135 for a short discussion.
2019-01-06 14:52:20 +01:00
Alexander Neumann
7c084014fa
Use processed bytes for summary
...
When restic reads the backup from stdin, the number of bytes processed
was always displayed as zero. The reason is that the UI for the archive
uses the total bytes as returned by the scanner, which is zero for
stdin. So instead we keep track of the real number of bytes processed
and print that at the end.
Closes #2136
2019-01-06 13:15:23 +01:00
Kevin Woo
d65bea1b2a
doc: Add more clarity to B2 application key
2019-01-05 20:05:11 -08:00
Andreas Olsson
3b68acf853
Prefer https:// links in the documentation
...
All the changed urls are available by way of https://. Most of them
already redirect.
2018-12-29 15:57:44 +01:00
James Culbertson
82a70643a2
Fix a few typos in the FAQ doc.
2018-12-18 08:47:52 -06:00
Alexander Neumann
0dd805421e
Merge pull request 2108 from mholt/noexitprint
...
cmd: Don't exit if printing to stdout or stderr fails (closes #2102 )
2018-12-15 21:34:18 +01:00
Alexander Neumann
16b82f4b1d
Merge pull request 2111 from andreaso/doc-moot-sudo
...
Remove moot sudo usage from documentation
2018-12-15 21:33:51 +01:00
Andreas Olsson
7a6bfcd58c
Remove moot sudo usage from documentation
...
That usage of sudo only applies to the echo command itself. It has no
effect on the permissions of the stdout redirect.
2018-12-09 17:43:50 +01:00
Matthew Holt
de54618852
cmd: Don't exit if printing to stdout or stderr fails ( closes #2102 )
2018-12-01 15:11:22 -07:00
Alexander Neumann
98526b8dbe
Merge pull request 2100 from restic/fix-1989
...
gs: Respect bandwidth limiting
2018-11-28 21:39:42 +01:00
Alexander Neumann
0083680d33
Merge pull request 2099 from restic/fix-1229
...
cache: Don't return an error for truncated files
2018-11-28 21:39:35 +01:00
Chris Howie
05222b7343
--one-file-system unsupported on Windows
2018-11-27 11:14:47 -05:00
Kyle Lacy
d4ff5b6bf4
Add changelog entry about "DefaultPermissions" change
2018-11-26 21:16:01 -08:00
Kyle Lacy
cf0883e16c
mount: Add "no-default-permissions" option
...
This option restores the previous behavior of `mount` by disabling the "DefaultPermissions" FUSE option. This allows any user that can access the mountpoint to read any file from the snapshot. Normal FUSE rules apply, so `allow-root` or `allow-other` can be used to allow users besides the mounting user to access these files.
2018-11-26 21:16:01 -08:00
Kyle Lacy
a35a24b8b4
mount: Enable "DefaultPermissions" FUSE option by default
...
This enforces the Unix permissions of the snapshot files within the mounted filesystem, which will only allow users to access snapshot files if they had access to the file outside of the snapshot.
2018-11-26 21:16:01 -08:00
Juergen Hoetzel
df7f72cdde
Add support for reading password from external command
...
This allows reading the password from an password manager (like "pass").
Signed-off-by: Juergen Hoetzel <juergen@archlinux.org>
2018-11-26 20:18:31 +01:00
plumbeo
3edc723bf0
Update 'restic forget --keep-within' documentation and add changelog entry
2018-11-26 14:27:48 +01:00
plumbeo
71891b340c
Support time ranges expressed in hours in snapshot retention policies
...
Make restic forget --keep-within accept time ranges measured in hours and choose
accordingly which snapshots to keep and which to forget. Add relative tests.
2018-11-26 14:27:42 +01:00
Alexander Neumann
6f5c3e57f6
Update vendor/modules.txt
2018-11-25 19:22:57 +01:00
Alexander Neumann
56af0ce370
Add entry to changelog
2018-11-25 19:07:42 +01:00
Alexander Neumann
c9745cd47e
gs: Respect bandwidth limiting
...
In 0dfdc11ed9
, accidentally we dropped
using the provided http.RoundTripper, this commits adds it back.
Closes #1989
2018-11-25 18:52:32 +01:00
Alexander Neumann
2434ab2106
Merge pull request 2098 from cdhowie/key-hinting
...
Add key hinting (#2097 )
2018-11-25 17:52:54 +01:00
Chris Howie
1688713400
Add key hinting ( #2097 )
2018-11-25 09:13:18 -05:00
Alexander Neumann
00597284de
cache: Don't return an error for truncated files
...
Closes #1229
Closes #1328
2018-11-25 14:18:02 +01:00
Johannes Hertenstein
879f6e0c81
Add changelog file for insensitive includes & excludes
2018-11-25 13:32:16 +01:00
Johannes Hertenstein
8a97bb8661
Add documentation for insensitive includes & excludes
2018-11-25 13:32:16 +01:00
Johannes Hertenstein
5fe6de219d
Add tests for insensitive variants of filter methods
2018-11-25 13:32:16 +01:00
Johannes Hertenstein
c13f79da02
Add iexclude and iinclude options to restore command
2018-11-25 13:32:16 +01:00
Johannes Hertenstein
db82e6b80c
Add iexclude option to backup command
2018-11-25 13:32:16 +01:00
Alexander Neumann
6dc7cca597
Merge pull request 2086 from frebib/fix-2085
...
Allow --files-from multiple times
2018-11-25 13:21:29 +01:00
Alexander Neumann
d32c7c2aba
Rework changelog for #2085
2018-11-25 12:55:03 +01:00
Joe Groocock
09e9b74cbd
Allow --files-from multiple times
...
Fixes https://github.com/restic/restic/issues/2085
2018-11-25 12:55:03 +01:00
Alexander Neumann
d53595e43c
Merge pull request 2088 from garrmcnu/stats-latest-host
...
stats: Add shorthand flag `-H` for host name filter
2018-11-24 19:05:53 +01:00
Alexander Neumann
0de19cc87f
Merge pull request 2095 from plumbeo/parse-location
...
Use local time for user-provided time of backup
2018-11-24 18:50:12 +01:00
plumbeo
2c9ec07d0b
Add changelog entry
2018-11-20 22:18:12 +01:00
plumbeo
a7971a3ece
Use local time for user-provided time of backup
2018-11-20 21:28:19 +01:00
Alexander Neumann
4ab0022da8
Add hint which OS are supported
...
Closes #2016
2018-11-18 12:33:52 +01:00
Garry McNulty
4b3c054257
stats: Add shorthand flag `-H` for host name filter
...
The default value of the `--host` flag was set to 'H' (the shorthand
version of the flag), this caused the snapshot lookup to fail.
Also add shorthand `-H` for `backup` command.
Closes #2040
2018-11-14 21:58:31 +00:00
plumbeo
7486bfea5b
Extend custom Duration granularity to hours and add tests
2018-11-14 17:27:02 +01:00
Alexander Neumann
c8fc72364a
Merge pull request 2081 from restic/fix-2080
...
backup: Open repository before async status starts
2018-11-11 14:21:47 +01:00
Alexander Neumann
987ef2f4a9
Merge pull request 2082 from restic/fix-2079
...
dump: Use path instead of filepath
2018-11-11 14:21:37 +01:00
Alexander Neumann
5b95bb7059
Fix error message on Windows
2018-11-11 14:06:32 +01:00
Alexander Neumann
8471a359ee
dump: Use path instead of filepath
...
Some time ago we changed the paths in the repo to always use a slash for
separation, it seems we missed that the `dump` command still uses the
`filepath` package, so on Windows backslashes are used.
Closes #2079
2018-11-10 12:43:11 +01:00
Alexander Neumann
f9422ff4c7
backup: Open repository before async status starts
...
Closes #2080
2018-11-10 12:38:29 +01:00
Alexander Neumann
c0572ca15f
backup: Add warning when patterns do not match any files
2018-11-03 14:51:30 +01:00
Alexander Neumann
a630d69e0c
Merge pull request 2070 from restic/display-local-time
...
Display local time for all commands
2018-11-02 21:16:29 +01:00
Alexander Neumann
20bcd281a3
Add entry to changelog
2018-11-02 20:40:52 +01:00
Alexander Neumann
c012fccd22
Display local time for all commands
2018-11-02 20:36:15 +01:00
Alexander Neumann
920727dd34
Merge pull request #2068 from restic/prune-check-load-error
...
prune: Correctly handle errors returned by Load()
2018-10-28 22:15:29 +01:00
Alexander Neumann
157d365894
Add entry to CHANGELOG
2018-10-28 21:38:04 +01:00
Alexander Neumann
bfa18ee8ec
DownloadAndHash: Check error returned by Load()
2018-10-28 21:28:56 +01:00
Alexander Neumann
890eebf151
Correct summary for #1978
2018-10-28 20:55:49 +01:00
Alexander Neumann
9310cd0cd6
Merge pull request #2056 from restic/recover-data
...
Add new command 'recover'
2018-10-28 14:26:48 +01:00
Alexander Neumann
9f7ce7ce5a
Merge pull request #2066 from restic/fix-list-error
...
rebuild-index/prune: Correctly handle error listing files
2018-10-28 14:26:40 +01:00
Alexander Neumann
0b600d6cef
index.New: Test various error conditions
2018-10-28 14:02:31 +01:00
Alexander Neumann
3ae2a79bdf
restic: Use local rand source instead of global one
2018-10-28 14:01:30 +01:00
Alexander Neumann
f7c0893f76
index: Add tests for error conditions
2018-10-28 13:16:36 +01:00
Alexander Neumann
c3de301fc8
index: Use interfaces instead of *repository.Repository
2018-10-28 12:58:34 +01:00
Alexander Neumann
944b446ac0
index: Cancel context for tests
2018-10-28 11:17:04 +01:00
Alexander Neumann
b096fc7abf
index: Correctly process errors listing all files
...
This also removes the now unused `list` and `worker` packages.
2018-10-28 11:16:29 +01:00
Alexander Neumann
d10754e2b4
Merge pull request 2057 from re1/patch-1
...
Update install instructions for Arch Linux
2018-10-21 20:04:07 +02:00
Alexander Neumann
7ac683c360
rclone: Inject debug logger for HTTP
2018-10-21 19:58:40 +02:00
Markus
6caa9d38ac
Update install instructions for Arch Linux
...
restic has been added to the official community repositories in august 2018.
2018-10-20 13:09:52 +00:00
Alexander Neumann
19fd0f101f
Merge pull request 2053 from andyjack/correct-doc-for-self-update
...
Fix incorrect command name in long doc for self-update
2018-10-20 11:51:13 +02:00
Alexander Neumann
8c91c51d1b
Fix timestamps
2018-10-20 11:08:03 +02:00
Alexander Neumann
7e28bf7e97
Add 'recover' command
2018-10-20 11:08:03 +02:00
Alexander Neumann
43d6e426c8
Merge pull request 2054 from gliptak/patch-3
...
Correct typo
2018-10-20 11:04:49 +02:00
Alexander Neumann
26fc60e7cb
Merge pull request 2055 from gliptak/patch-4
...
Correct ineffassign
2018-10-20 11:04:32 +02:00
Gábor Lipták
e5d7879622
Correct ineffassign
...
Signed-off-by: Gábor Lipták <gliptak@gmail.com>
2018-10-19 16:58:14 -04:00
Gábor Lipták
d2ee58f2e9
Correct typo
2018-10-18 21:30:14 -04:00
Andy Jack
3f25537a06
Fix incorrect command name in long doc for self-update
2018-10-18 18:45:31 -04:00
Alexander Neumann
d203ae37f4
Merge pull request 2050 from andreasnuesslein/patch-1
...
tiny formatting fix
2018-10-18 22:31:15 +02:00
Andreas Nüßlein
6eedd66c1a
tiny formatting fix
...
I noticed this discrepancy when trying to parse the output of restic.
2018-10-17 09:41:11 +02:00
Alexander Neumann
e4b39ae553
Merge pull request 2044 from restic/fix-verify
...
restore: Close files tested with --verify
2018-10-15 21:16:58 +02:00
Alexander Neumann
7cbcb6d318
Merge pull request 2043 from cgroschupp/patch-1
...
Add ssh binary to docker image to allow sftp repository
2018-10-14 21:43:04 +02:00
Alexander Neumann
c0fca3f50a
restore: Close files tested with --verify
...
Before, the target files were opened once per blob and never closed,
this commit fixes that.
2018-10-14 21:00:14 +02:00
Alexander Neumann
4c2072d875
Merge pull request 1719 from ifedorenko/mt-restorer
...
new optimized multithreaded restore implementation
2018-10-14 20:29:16 +02:00
Christian Groschupp
92ecca1808
Add ssh binary to docker image to allow sftp repository
2018-10-14 19:56:02 +02:00
Alexander Neumann
7236635cc1
Merge pull request 2042 from restic/fix-2041
...
self-update: fix path to output file
2018-10-14 19:48:28 +02:00
Alexander Neumann
21a3486ebb
Rework changelog entry
2018-10-14 17:39:42 +02:00
Igor Fedorenko
bda8d7722e
restorer: Optimize empty file restore
...
don't create fileInfo structs for empty files. this saves memory.
this also avoids extra serial scan of all fileInfo, which should
make restore faster and more consistent.
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:39:42 +02:00
Igor Fedorenko
c2bcb764cd
restorer: Add a note on hardlink metadata
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:39:42 +02:00
Igor Fedorenko
9e24154ec9
restore: significantly reduce memory footprint
...
reworked restore error callback to use file location
path instead of much heavier Node. this reduced restore
memory usage by as much as 50% in some of my tests.
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:39:42 +02:00
Igor Fedorenko
9f3ca97ee8
restore: Chang fileInfo to use snapshot location instead of target path
...
* uses less memory as common prefix is only stored once
* stepping stone for simpler error callback api, which
will allow further memory footprint reduction
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:39:42 +02:00
Igor Fedorenko
32d5ceba87
restore: Fix packcache capacity math with failed downloads
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:39:42 +02:00
Igor Fedorenko
e010f3b884
restore: Fix small memory leak in filesWriter, add tests
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:39:42 +02:00
Alexander Neumann
941202c119
restore: Rename things, match the rest of the project
2018-10-14 17:39:37 +02:00
Alexander Neumann
c021ad2334
restore: Move documentation to godoc
2018-10-14 17:39:35 +02:00
Igor Fedorenko
2b3420820b
go mod vendor
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:37:30 +02:00
Igor Fedorenko
da57302fca
restore: Removed legacy restore implementation
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:37:30 +02:00
Igor Fedorenko
1869930d95
restore: New optimized multithreaded implementation
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:37:30 +02:00
Igor Fedorenko
1213d8fef4
Cleanup: more realistic restorer test data setup
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-10-14 17:37:30 +02:00
Alexander Neumann
a432b42c81
self-update: Check current version before download
2018-10-14 17:29:46 +02:00
Alexander Neumann
7d0f2eaf24
self-update: Use correct path to restic binary
...
Closes #2041
2018-10-14 17:29:19 +02:00
Alexander Neumann
41a4d67d93
Merge pull request 1962 from mholt/lsjson
...
ls: Stream output when using --json option
2018-10-14 15:51:38 +02:00
Alexander Neumann
afde60e433
Merge pull request #2039 from restic/fix-2038
...
snapshots: Don't print empty line for --compact
2018-10-13 21:24:26 +02:00
Alexander Neumann
d7baa67acb
snapshots: Don't print empty line for --compact
...
Closes #2038
2018-10-13 20:17:34 +02:00
Alexander Neumann
167397c18c
Set development version for 0.9.3
2018-10-13 13:48:31 +02:00
Alexander Neumann
be36c5f150
Add version for 0.9.3
2018-10-13 13:48:31 +02:00
Alexander Neumann
9484a14ab2
Update manpages and auto-completion
2018-10-13 13:48:30 +02:00
Alexander Neumann
0f5fc8fb3d
Generate CHANGELOG.md for 0.9.3
2018-10-13 13:48:29 +02:00
Alexander Neumann
a5b40e9372
helpers: Use version string in built binaries
2018-10-13 13:48:03 +02:00
Alexander Neumann
c5ec4efe91
Update prepare-release
2018-10-13 13:48:03 +02:00
Alexander Neumann
e64a0e0454
helpers: Don't run sha256sums in build
2018-10-13 13:48:03 +02:00
Alexander Neumann
8b5b031f90
Prepare changelog for 0.9.3
2018-10-13 13:48:03 +02:00
Alexander Neumann
4a2134bbc5
changelog: Rename issue
2018-10-13 13:48:03 +02:00
Alexander Neumann
484844aa1a
Document the build and release processes
2018-10-13 13:48:03 +02:00
Alexander Neumann
4ed10239ad
doc: Add developer information
2018-10-11 22:37:25 +02:00
Alexander Neumann
c4896ed642
Add build-release-binaries
2018-10-11 22:37:25 +02:00
Alexander Neumann
29aaec383c
Move scripts/ to helpers/
2018-10-11 22:37:25 +02:00
Alexander Neumann
0cb241b7d3
Remove build tag `release`
2018-10-11 19:40:38 +02:00
Alexander Neumann
de4750b8e0
Add entry to changelog
2018-10-11 19:37:20 +02:00
Alexander Neumann
7b91c40e21
Merge pull request #2033 from j6s/feature.cache-size
...
Output directory size in cache command
2018-10-11 19:34:57 +02:00
Alexander Neumann
cc9bf02da1
Merge pull request #2036 from SimJoSt/patch-1
...
doc: cache: fix typo
2018-10-11 19:34:14 +02:00
Joda Stößer
b7959c44d2
doc: cache: fix typo
2018-10-11 14:00:51 +02:00
Johannes Hertenstein
277cba4b32
Catch errors when walking cache directories
2018-10-08 15:47:34 +02:00
Johannes Hertenstein
ed651df19b
Use correct method for joining paths
2018-10-08 15:47:08 +02:00
Johannes Hertenstein
641dc65e6e
Output directory size in cache command
2018-10-07 14:37:51 +02:00
Alexander Neumann
de9136b29f
Merge pull request #2022 from moritzdietz/update-faq
...
Add example to spot path expansion errors
2018-10-07 12:11:13 +02:00
Moritz Dietz
b36345fd84
Add example to spot path expansion errros
2018-10-07 12:09:46 +02:00
Alexander Neumann
03402c8a04
Merge pull request #2031 from danielb2/latest
...
use latest instead of newest in forget language
2018-10-06 21:20:21 +02:00
Daniel Bretoi
966e5a5575
use latest instead of newest in forget language
2018-10-06 12:02:22 -07:00
Alexander Neumann
5aa0deeff9
Merge pull request #2026 from gottwald/update-go4docker
...
Update Go version to 1.11.1 in Docker build script
2018-10-06 13:11:19 +02:00
Alexander Neumann
af4d822380
Merge pull request #2025 from kurin/fixloop
...
b2: simplify object iteration
2018-10-06 13:01:27 +02:00
Alexander Neumann
fd95b86894
Merge pull request #2029 from j6s/feature.escape-docker-build
...
Escape subcommand in docker build script
2018-10-06 12:46:08 +02:00
Alexander Neumann
5dbef3712e
Merge pull request #2027 from j6s/feature.cache-dir-information
...
Print base directory with cache command
2018-10-06 12:40:48 +02:00
Johannes Hertenstein
63647e93e4
Escape subcommand in docker build script
2018-10-05 22:13:49 +02:00
Johannes Hertenstein
9b8deb51ba
Print base directory with cache command
2018-10-05 21:23:57 +02:00
Ingo Gottwald
2c4b0d975e
Update Go version to 1.11.1 in Docker build script
...
Restic cannot be built with Go 1.8.3 any more, it requires at least Go
1.9.0.
2018-10-05 21:13:28 +02:00
Toby Burress
8ceda538ef
b2: simplify object iteration
...
Blazer is moving to a simpler object list interface, so I'm changing
this here as well.
2018-10-05 11:39:02 -07:00
Alexander Neumann
233596f4bc
Merge pull request #2019 from restic/recheck-cache
...
cache: Recheck before downloading
2018-10-05 12:26:21 +02:00
Alexander Neumann
6712ee8f92
Merge pull request #2020 from j6s/feature.cache-dir-documentation
...
Add defaults to command line help of --cache-dir
2018-10-05 12:26:12 +02:00
Johannes Hertenstein
0916ff71bd
Add defaults to command line help of --cache-dir
2018-10-04 19:50:05 +02:00
Alexander Neumann
5971650f77
cache: Fix recheck logic, remove channel from inProgress
2018-10-04 17:09:43 +02:00
Alexander Neumann
19725954ee
cache: Recheck before downloading
2018-10-04 14:31:18 +02:00
Alexander Neumann
b1e1b71bab
docker: Base image on latest alpine release
2018-10-03 21:33:13 +02:00
Alexander Neumann
f1799de309
Merge pull request #2018 from restic/fix-1967
...
Use `--host` for all commands, deprecate --hostname
2018-10-03 21:19:01 +02:00
Alexander Neumann
585a5e3416
Use `--host` for all commands, deprecate --hostname
2018-10-03 14:12:35 +02:00
Alexander Neumann
b7eeeedc3f
Add minimal docs for the top-level package
...
Closes #173
2018-10-03 13:43:40 +02:00
Alexander Neumann
a20d4bc6b0
Merge pull request #1892 from Stell0/1891
...
Expand Glob (wildcards character) in paths in file in --files-from
2018-10-03 12:05:55 +02:00
Alexander Neumann
fb31d66951
backup: Improve error message for invalid pattern
2018-10-03 11:12:51 +02:00
Alexander Neumann
33dfbf5c38
Reword changelog
2018-10-03 10:37:35 +02:00
Stefano Fancello
d1df3718b5
Add changelog unreleased file
2018-10-03 10:37:35 +02:00
Stefano Fancello
e2da0a416c
Expand Glob (wildcards character) in paths in file in --files-from
2018-10-03 10:37:35 +02:00
Alexander Neumann
0c0a8e3d2b
docs: Explain recognizing modified files
2018-10-02 14:21:08 +02:00
Alexander Neumann
0882aca3a8
Add hint for FUSE on FreeBSD
2018-09-26 21:40:43 +02:00
Alexander Neumann
cd41915e10
Merge pull request #2009 from restic/update-minio
...
Update github.com/minio/minio-go
2018-09-26 14:33:30 +02:00
Alexander Neumann
2effacd444
Update github.com/minio/minio-go
...
Closes #1918
2018-09-26 14:12:34 +02:00
Alexander Neumann
c6901ff908
Merge pull request #2002 from ifedorenko/dumb-terminals
...
termstatus: detect and respect dumb terminals on Unix
2018-09-25 13:53:57 +02:00
Alexander Neumann
2f774acce3
Merge pull request #2006 from Calinou/doc-add-scoop-installation
...
Add Scoop installation method to the documentation
2018-09-25 13:23:28 +02:00
Alexander Neumann
5f8658238c
Merge pull request #2005 from Calinou/doc-osx-to-macos
...
Rename OS X to macOS in the documentation
2018-09-25 13:21:28 +02:00
Hugo Locurcio
2bb1be4d4e
Add Scoop installation method to the documentation
2018-09-23 18:42:50 +02:00
Hugo Locurcio
40e0016403
Rename OS X to macOS in the documentation
...
macOS is the official name since the release of macOS 10.12 (Sierra).
2018-09-23 16:13:34 +02:00
Igor Fedorenko
541d232f1c
termstatus: detect and respect dumb terminals on Unix
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-09-21 07:24:41 -04:00
Alexander Neumann
6bc99ce451
Merge pull request #1993 from rfjakob/lchown
...
restore: suppress lchown errors when not running as root
2018-09-09 20:10:50 +02:00
Jakob Unterwurzacher
e42d2d1da8
restore: suppress lchown errors when not running as root
...
Like "cp -a" and "rsync -a" do, only report lchown errors if we run
as root.
Like cp from GNU coreutils does, we check Geteuid() to determine if
we are running as root
( http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/copy.c#n3012 ).
On Windows, lchown errors are always reported.
Fixes https://github.com/restic/restic/issues/1766
2018-09-09 15:39:10 +02:00
Alexander Neumann
bd9022962e
Merge pull request #1992 from restic/add-go-mod-tidy
...
Run go mod tidy during CI
2018-09-09 11:03:16 +02:00
Alexander Neumann
91f1b40206
Run go mod tidy during CI
2018-09-09 10:36:27 +02:00
Alexander Neumann
d9b89eead0
Run 'go mod tidy'
2018-09-08 22:12:23 +02:00
Alexander Neumann
5399297de6
Merge pull request #1980 from restic/update-deps
...
Update dependencies
2018-09-08 21:57:39 +02:00
Alexander Neumann
96f7be5d9b
Update dependencies
2018-09-08 21:17:22 +02:00
Alexander Neumann
0922367308
Merge pull request #1991 from restic/debug-1978
...
scanner: Use context only for cancellation
2018-09-08 20:22:15 +02:00
Alexander Neumann
e2d9900d82
Add entry to CHANGELOG
2018-09-08 18:53:12 +02:00
Alexander Neumann
1140950d7b
scanner: Use context only for cancellation
...
When the scanner is slower than the actual backup, the tomb cancels the
context passed to Scan(), which then returns ctx.Err(). In the end, the
main function prints an error message that is not helpful ("Context
cancelled") and exits with an error code although no error occurred.
The code now ignores the error in the context and just uses it for
cancellation. The scanner is not supposed to return an error anyway.
Closes #1978
2018-09-08 18:53:12 +02:00
Alexander Neumann
6d9c008900
Add template for "regular" issue
2018-09-08 18:00:29 +02:00
Alexander Neumann
b617444158
Test issue template
2018-09-08 17:58:08 +02:00
Alexander Neumann
e588c42646
Move template
2018-09-08 17:55:44 +02:00
Alexander Neumann
14bb2a9005
Add custom issue template
2018-09-08 17:53:52 +02:00
Alexander Neumann
f04d347e7a
Merge pull request #1982 from tyll/installation
...
Update installation instructions
2018-09-05 21:15:38 +02:00
Alexander Neumann
746182c526
Correct name for Apple's desktop OS
2018-09-05 20:41:37 +02:00
Alexander Neumann
08beb7d84c
Add section about FreeBSD, move anchor
2018-09-05 20:40:28 +02:00
Alexander Neumann
9795b00f51
Merge pull request #1983 from armhold/vet-cleanup
...
gh-1385 clean up some errors from 'go vet ./...'
2018-09-05 20:31:11 +02:00
George Armhold
bfc1bc6ee6
clean up some errors from 'go vet ./...'
2018-09-05 08:04:55 -04:00
Till Maas
e9cdcf131c
Update installation instructions
...
- sort sections for different third-party methods
- Mention that Fedora can now just be installed via dnf
- Fix some typos
2018-09-04 14:39:40 +02:00
Alexander Neumann
35e9885e8b
Appveyor: correct build cache location
2018-09-02 13:26:02 +02:00
Alexander Neumann
16885529f7
Merge pull request #1973 from restic/announce-new-cache
...
cache: Print message when new cache is created
2018-09-02 13:00:55 +02:00
Alexander Neumann
3c02eeb5a8
Merge pull request #1920 from restic/support-go111
...
Support Go 1.11
2018-09-01 11:38:41 +02:00
Alexander Neumann
9e9bb62ad4
Travis: Don't use cached test results
2018-08-31 22:00:47 +02:00
Alexander Neumann
175e630717
Travis: Also cache $GOPATH/pkg/mod
2018-08-31 22:00:47 +02:00
Alexander Neumann
44f38ad049
Travis: Also cache on Darwin
2018-08-31 22:00:47 +02:00
Alexander Neumann
ca928aeae4
Appveyor: Cache Go build cache
2018-08-31 22:00:47 +02:00
Alexander Neumann
27b60a05b4
Travis: Cache Go build artifacts
2018-08-31 22:00:47 +02:00
Alexander Neumann
8af4b331ef
Travis: Remove gotestcover
2018-08-31 22:00:47 +02:00
Alexander Neumann
a5a46e4989
Travis: don't use GOPATH for Go >= 1.11
2018-08-31 22:00:47 +02:00
Alexander Neumann
e4cdb0eab3
Travis: Run with -mod=vendor only for Go >= 1.11
2018-08-31 22:00:47 +02:00
Alexander Neumann
e9a764129f
Update docs for Go 1.11
2018-08-31 22:00:47 +02:00
Alexander Neumann
65129bde5e
Makefile: Try -mod=vendor first
2018-08-31 22:00:47 +02:00
Alexander Neumann
b4beaf807b
Travis: Run build.go with -mod=vendor
2018-08-31 22:00:47 +02:00
Alexander Neumann
4734056583
build.go: Major rework, support Go modules
2018-08-31 22:00:47 +02:00
Alexander Neumann
71e0408390
Add entry to changelog
2018-08-31 21:10:26 +02:00
Alexander Neumann
1352a9d848
run_integration_tests: Replace dep with Go modules
2018-08-31 21:10:26 +02:00
Alexander Neumann
e0f68ec2c0
Vendor dependencies with `go mod vendor`
2018-08-31 21:10:26 +02:00
Alexander Neumann
9c6e0c6eb9
Document gofmt version
2018-08-31 21:10:26 +02:00
Alexander Neumann
4cbc7c4467
Only run gofmt for latest stable version
2018-08-31 21:10:26 +02:00
Alexander Neumann
aaff8803ef
Fix formatting for gofmt with Go 1.11
2018-08-31 21:10:26 +02:00
Alexander Neumann
16e20676b6
build.go: Set GOPROXY=off
2018-08-31 21:10:26 +02:00
Alexander Neumann
6cd5f8b7f5
Set GOXPROXY=off for tests, run with -mod=vendor
2018-08-31 21:10:26 +02:00
Alexander Neumann
10c0b8080e
Add Go 1.11 to CI tests
2018-08-31 21:10:26 +02:00
Alexander Neumann
d31666d332
build.go: Don't ignore error
2018-08-31 19:51:48 +02:00
Alexander Neumann
6d53e767d5
cache: Print message when new cache is created
...
Sometimes, users run restic without retaining the local cache
directories. This was reported several times in the past.
Restic will now print a message whenever a new cache directory is
created from scratch (i.e. it did not exist before), so users have a
chance to recognize when the cache is not kept between different runs of
restic.
2018-08-29 22:01:10 +02:00
Alexander Neumann
f1b0bb33dd
Merge pull request #1975 from moritzdietz/fix-env-docu
...
Update B2 env variable information
2018-08-29 22:00:17 +02:00
Moritz Dietz
99ae913414
Update B2 env variable information
...
* Fixed the B2 environment variables as they had the same description
* Added the variables for the newly introduced Application Key support #1906
2018-08-29 21:05:57 +02:00
Thomas Damgaard
df78896e59
doc: Add list of environment variables
2018-08-28 22:18:47 +02:00
Alexander Neumann
c896751ce2
Merge pull request #1970 from McKael/pull-1780-changelog
...
Improve changelog entry (pull-1780)
2018-08-28 20:59:10 +02:00
Alexander Neumann
501189625e
Merge pull request #1971 from bobsaintcool/doc/fix_man_output
...
Doc/fix man output
2018-08-28 20:58:21 +02:00
Quentin Bourgeois
a065ada46a
Make generated man pages more easy to read with items
...
Allow Cobra to output a more list items friendly format, this make
reading of the documentation more easily using information from
manpages.
2018-08-27 17:42:54 +02:00
Mikael Berthe
17d6d537e2
Improve changelog entry (pull-1780)
2018-08-27 15:58:39 +02:00
Quentin Bourgeois
5cc224e44a
Merge remote-tracking branch 'upsteam/master'
2018-08-26 20:05:02 +02:00
Alexander Neumann
896089976a
Fix comment
2018-08-26 19:54:17 +02:00
Quentin Bourgeois
a563f87818
<docs/manual_rest: Fix broken link to restc Design page>
2018-08-26 01:46:38 +02:00
Alexander Neumann
de307ea2ab
Merge pull request #1876 from restic/forget-explain
...
forget: Add --explain
2018-08-25 21:48:44 +02:00
Alexander Neumann
4bc904a527
Merge pull request #1780 from McKael/what-contains
...
Add options to 'find' for searching IDs (blobs, ...)
2018-08-25 21:46:25 +02:00
Alexander Neumann
5937b5b355
Add entry to changelog
2018-08-20 22:00:23 +02:00
Alexander Neumann
76387b6cd0
Remove old text table implementation
2018-08-20 22:00:23 +02:00
Alexander Neumann
9aa36a37c7
Vendor cmpopts
2018-08-20 21:47:51 +02:00
Alexander Neumann
9fd3796d93
forget: Display reasons why snapshots are kept
...
This change displays the reasons for keeping a snapshot in the table,
unless `--compact` is specified.
2018-08-20 21:47:51 +02:00
Mikael Berthe
93fa17b53f
Add entry to changelog (new find flags)
2018-08-19 23:28:04 +02:00
Alexander Neumann
15ad0e5bc7
walk: Pass parent tree ID to WalkFunc
2018-08-19 23:28:04 +02:00
Mikael Berthe
1f27d17c0d
walker.Walk: Pass parent tree-id to WalkFunc
2018-08-19 23:28:04 +02:00
Mikael Berthe
8af918a1e4
find: Add support for multiple patterns or objects
2018-08-19 23:28:04 +02:00
Mikael Berthe
bb5425a1d8
find: Add support for blobs, IDs and packs lookups
...
With --blob, --tree and --pack, the find command now lists the snapshots
that contain a specific tree or blob, or the snapshots that contain
blobs belonging to a given pack.
It also displays the pack ID a blob belongs to.
A list of IDs can be given, as long as the IDs are all of the same type.
2018-08-19 22:32:30 +02:00
Alexander Neumann
12246969db
ui/table: Add small package for writing tables
2018-08-19 21:39:35 +02:00
Matthew Holt
9151eec24e
Add changelog entry
2018-08-19 09:14:58 -06:00
Matthew Holt
22475729ce
Select specific Node fields for listing
2018-08-19 09:03:47 -06:00
Matthew Holt
04c67d700d
ls: Stream output when using --json option
2018-08-19 00:18:43 -06:00
Alexander Neumann
d708d607fa
Merge pull request #1953 from kitone/ls-add-json
...
ls: Add JSON output support for restic ls cmd
2018-08-18 15:31:15 +02:00
Alexander Neumann
46f71f4c22
Improve changelog entry
2018-08-18 15:14:52 +02:00
kitone
48cc2f2188
fix: switch struct_type value to lower case
2018-08-18 14:57:50 +02:00
kitone
bd6e7c934c
add changelog entry
2018-08-18 14:57:50 +02:00
kitone
7925217e25
ls: Add JSON output support for restic ls cmd
2018-08-18 14:57:50 +02:00
Alexander Neumann
401a564486
Merge pull request #1961 from mholt/statsoutput
...
stats: Show what was scanned and scanning mode used
2018-08-18 14:29:14 +02:00
Alexander Neumann
31176d212b
doc: Add explanation for "processed" vs "added"
2018-08-18 14:26:57 +02:00
Matthew Holt
2d89311d49
stats: Show what was scanned and scanning mode used
2018-08-17 17:15:30 -06:00
Alexander Neumann
5a25ad1972
Update version in build.go
2018-08-17 21:17:26 +02:00
Alexander Neumann
79d3a18b31
release.go: Add version to global.go
...
Closes #1958
2018-08-17 21:17:10 +02:00
Alexander Neumann
89f17847ad
Merge pull request #1955 from restic/fix-prune-1954
...
prune: Fix calculation for removed bytes
2018-08-14 22:30:59 +02:00
Alexander Neumann
1ab5703404
prune: Fix calculation for removed bytes
2018-08-14 22:06:05 +02:00
Alexander Neumann
49d95e9a50
Merge pull request #1949 from restic/add-self-update
...
Add command self-update
2018-08-13 22:02:34 +02:00
Alexander Neumann
7dff1a08d0
Merge pull request #1950 from ldelouw/master
...
Add RPM SPEC file to create restic packages on Fedora and RHEL
2018-08-13 22:00:18 +02:00
Alexander Neumann
5fee36fa84
Merge pull request #1941 from mholt/lsfilter
...
ls: Implement directory filter, optionally subfolders
2018-08-13 21:58:32 +02:00
Luc de Louw
b0211dff49
Small change for RHEL6 which does not know about the %license macro
2018-08-13 21:39:04 +02:00
Luc de Louw
0f6d21cf84
Add the SPEC file restic.spec to build RPMs
...
Tested on Fedora 28, RHEL 6 and 7
2018-08-13 21:39:04 +02:00
Alexander Neumann
10b5cf8f32
Add self-update to the docs
2018-08-12 23:56:06 +02:00
Alexander Neumann
ad5aec3f3b
Add entry to changelog
2018-08-12 23:56:01 +02:00
Alexander Neumann
6e1a3987b7
Add 'self-update' command
...
This commit adds a command called `self-update` which downloads the
latest released version of restic from GitHub and replacing the current
binary with it. It does not rely on any external program (so it'll work
everywhere), but still verifies the GPG signature using the embedded GPG
public key.
By default, the `self-update` command is hidden behind the `selfupdate`
built tag, which is only set when restic is built using `build.go`. The
reason for this is that downstream distributions will then not include
the command by default, so users are encouraged to use the
platform-specific distribution mechanism.
2018-08-12 23:34:47 +02:00
Alexander Neumann
9630398e3b
ls: Rework and simplify logic
...
This commit introduces two functions: withinDir() and
approachingMatchingTree()
Both bind the list of directories with a closure, so we don't need to
iterate over the list in the function passed to Walk(). This reduces the
indentation level and since we can just use return, we don't need the
breaks any more.
The case that len(dirs) == 0 can also be handled by the functions with a
return, which saves another indentation level.
The main function body of the function passed to Walk() was reduced to
three cases:
* Within one of the dirs: Print the node, and if recursive operation is
requested, directly return, so the walker continues recursive
traversal
* Approaching one of the dirs: don't print anything, but continue
recursive traversal.
* Nothing of the two: abort walking this branch of the tree.
2018-08-12 23:13:34 +02:00
Alexander Neumann
7e34de4c29
ls: Add comments
2018-08-12 22:18:44 +02:00
Alexander Neumann
ace5cc4ed3
ls: Only skip directory nodes
...
Special case for Walk(): When SkipDir is returned for a non-dir node,
the remaining nodes for the current tree are skipped. We don't want
that.
2018-08-12 22:02:59 +02:00
Alexander Neumann
7f617cfd7f
ls: Use nodepath for filter
2018-08-12 22:01:38 +02:00
Alexander Neumann
0deb4e5994
ls: Check dirs before opening the repository
...
Users get feedback instantly, and before any expensive network calls
have been made.
2018-08-12 21:59:57 +02:00
Alexander Neumann
6b9dde3ce8
Merge pull request #1912 from askielboe/select-funcs
...
Support for different kinds of select functions
2018-08-12 19:26:36 +02:00
Alexander Neumann
c145b618d4
Add entry to changelog
2018-08-12 17:51:12 +02:00
Andreas Skielboe
b07bb3d8c3
Reject files excluded by name before calling lstat to improve scan speed
...
Adds a SelectByName method to the archive and scanner which only require
the filename as input, and can thus be run before calling lstat on the
file. Can speed up scanning significantly if a lot of filename excludes
are used.
2018-08-12 17:51:12 +02:00
Alexander Neumann
9b513312e2
Merge pull request #1946 from restic/fix-1945
...
Remove truncated files from cache
2018-08-12 17:23:56 +02:00
Alexander Neumann
bf26a3ed57
Merge pull request #1948 from restic/update-build-go
...
Update build.go
2018-08-12 17:18:11 +02:00
Alexander Neumann
77a8d931b8
Update build.go
2018-08-12 15:44:13 +02:00
Matthew Holt
11ce572894
Fix bug where some folder listings were empty
2018-08-11 17:17:43 -06:00
Matthew Holt
7a468d1226
Speed up nonrecursive queries; include exact filter match
2018-08-11 16:18:09 -06:00
Matthew Holt
00e2fd8b5f
Apply feedback and use SkipNode
2018-08-11 15:25:22 -06:00
Alexander Neumann
0f83fea007
cache: Fix test for new behavior
...
Accessing beyond the end of the file now removes the file from the cache
because it is assumed to be truncated. Usually, this means that the data
is fetched directly from the backend instead.
2018-08-11 23:11:51 +02:00
Alexander Neumann
04f7c054cd
Add entry to changelog
2018-08-11 22:54:22 +02:00
Alexander Neumann
5dd0df0162
cache: Remove files from cache which are too small
2018-08-11 22:47:01 +02:00
Alexander Neumann
abc923f693
Merge pull request #1942 from mholt/statserr
...
stats: Improve error message for bad snapshot ID (fixes #1933 )
2018-08-11 14:26:24 +02:00
Matthew Holt
ac3bd6b2eb
Replace Exitf with errors.Fatalf
2018-08-10 22:15:33 -06:00
Matthew Holt
156d85a29b
Minor fixes/tweaks; add docs
2018-08-10 22:10:02 -06:00
Matthew Holt
8c146eac4b
ls: Implement directory filter, optionally subfolders
2018-08-10 21:41:38 -06:00
Matthew Holt
6f5b0f3622
stat: Improve error message for bad snapshot ID ( fixes #1933 )
2018-08-08 16:49:36 -06:00
Alexander Neumann
beb208e159
Add VERSION for 0.9.2
2018-08-06 21:28:59 +02:00
Alexander Neumann
c221d662d0
Update manpages and auto-completion
2018-08-06 21:28:59 +02:00
Alexander Neumann
143597d445
Generate CHANGELOG.md for 0.9.2
2018-08-06 21:28:48 +02:00
Alexander Neumann
16ca837763
Create changelog/ release subdir for 0.9.2
2018-08-06 21:28:16 +02:00
Alexander Neumann
ce7fb166b3
Merge pull request #1927 from prosoitos/minor-fixes_examples
...
minor-fixes_examples
2018-08-05 12:27:03 +02:00
Marie-Helene Burle
9de51d04ec
minor-fixes_examples
...
- update link to latest restic release to install restic for `restic` user
- correct typo in path to this app
2018-08-05 02:15:17 -07:00
Alexander Neumann
dc39773cd2
Merge pull request #1924 from cuspat96/master
...
Clarify B2 Application Keys once again
2018-08-03 21:15:16 +02:00
cuspat96
30fa305c07
Clarify B2 Application Keys once again
...
This commit fixes documentation about Application Keys in Backblaze B2.
2018-08-02 21:29:29 +02:00
Alexander Neumann
686f24b578
doc: Clarify B2 application keys
2018-08-02 21:14:05 +02:00
Alexander Neumann
247d2b7215
Merge pull request #1921 from salbertson/patch-1
...
Add a "Reviewed by Hound" badge
2018-08-02 20:03:43 +02:00
Alexander Neumann
017cd113d3
Merge pull request #1922 from salbertson/patch-2
...
Use https when linking to chris.beams.io
2018-08-02 20:03:40 +02:00
Scott Albertson
f744c2553e
Use https when linking to chris.beams.io
...
Why not link to How to [Write a Git Commit Message](https://chris.beams.io/posts/git-commit/ ) using HTTPS, it's going to redirect anyway.
2018-08-01 14:59:06 -07:00
Alexander Neumann
56cd6bd495
Merge pull request #1919 from restic/update-deps
...
Update dependencies
2018-08-01 23:56:55 +02:00
Alexander Neumann
bff635bc5f
Update dependencies, enable pruning for vendor/
...
So, `dep` got an nice new feature to remove tests and non-go files from
`vendor/`, and this brings the size of the vendor directory from ~300MiB
down to ~20MiB. We don that now.
2018-08-01 21:32:15 +02:00
Alexander Neumann
3422c1ca83
Merge pull request #1729 from mholt/stats
...
Implement `restic stats` command to get more info about a repository
2018-07-31 23:24:36 +02:00
Matthew Holt
f6b2731aa5
stats: Add manual doc, improve -h doc
...
Also rename files-by-content to files-by-contents, once and for all
2018-07-31 22:54:10 +02:00
Scott Albertson
3eb5b45b41
Add a "Reviewed by Hound" badge
2018-07-31 13:53:24 -07:00
Alexander Neumann
01aacf41b5
Merge pull request #1915 from mlissner/patch-2
...
Adds warning re performance of prune
2018-07-31 22:42:20 +02:00
Mike Lissner
2caf8edc55
Add warning of the performance of prune
...
I went pretty loud with this, but I think the performance is bad enough
that it's really worth highlighting, especially since it locks the index
during the prune.
2018-07-31 22:41:40 +02:00
Alexander Neumann
3151978f58
Fix changelog type
2018-07-31 21:57:27 +02:00
Alexander Neumann
ab4ef432ff
Add entry to changelog
2018-07-31 21:29:47 +02:00
Alexander Neumann
be4f54b603
Merge pull request #1913 from restic/restic-password-stdin-message
...
Print message for password being read from stdin
2018-07-31 21:28:12 +02:00
Alexander Neumann
7260110c27
Merge pull request #1914 from restic/update-blazer
...
Add support for B2 application keys
2018-07-31 21:27:50 +02:00
Alexander Neumann
2437f11af7
Update github.com/kurin/blazer to 0.5.1
...
This adds support for B2 application keys.
2018-07-31 20:51:36 +02:00
Alexander Neumann
57873502f8
Add note about B2 application keys to the docs
2018-07-31 20:49:54 +02:00
Alexander Neumann
3678ec9ad8
Print message for password being read from stdin
...
Closes #1911
2018-07-31 20:21:18 +02:00
Alexander Neumann
a717e9e6f7
Improve message for number of bytes newly added
2018-07-31 19:08:43 +02:00
Alexander Neumann
12c797700e
make statsWalkSnapshot return a function
2018-07-27 21:44:59 +02:00
Matthew Holt
daca9d6815
Consolidate mode flags; use new Walk function
2018-07-27 21:27:40 +02:00
Matthew Holt
930602a444
Update comment now that question was answered
2018-07-27 21:27:39 +02:00
Matthew Holt
acb05e7855
Fix filepath uniqueness bug for blobs-per-file mode
2018-07-27 21:27:39 +02:00
Matthew Holt
a7b95d716a
Implement four counting modes
2018-07-27 21:27:39 +02:00
Matthew Holt
925b542eb0
Count unique files by blob sequence rather than tree ID
2018-07-27 21:27:39 +02:00
Matthew Holt
f7659bd8b0
stats: Initial implementation of stats command
2018-07-27 21:27:39 +02:00
Alexander Neumann
8c124a2b75
Merge pull request #1902 from mlissner/patch-1
...
b2 bucket names need to be unique
2018-07-23 22:58:42 +02:00
Mike Lissner
d3ad63a4ec
b2 bucket names need to be unique
...
Adds a small warning indicating that b2 bucket names need to be unique. It's an easy mistake to make, and it's surprising to get the following error if you're not accustomed to the way B2 works:
Fatal: create repository at b2:postgres failed: NewBucket: b2_create_bucket: 400: Bucket name is already in use
2018-07-23 11:48:59 -07:00
Alexander Neumann
271c50cf5c
Add entry to changelog
2018-07-23 20:15:55 +02:00
Alexander Neumann
1aeb193fd9
Merge pull request #1900 from restic/fix-1870
...
restorer: Add test for restore with include filter
2018-07-23 20:15:50 +02:00
Alexander Neumann
f715bef82f
Merge pull request #1899 from garrmcnu/check-cache-dir
...
check: Use --cache-dir argument
2018-07-22 21:03:52 +02:00
Alexander Neumann
4fc00d4120
Merge pull request #1901 from restic/update-blazer
...
Update github.com/kurin/blazer
2018-07-22 20:59:52 +02:00
Garry McNulty
7603ab7ac1
check: Update --cache-dir argument handling based on code review comments
...
The temporary cache directory is created in the specified directory, or
if not specified in the default temporary directory.
2018-07-22 18:24:11 +01:00
Alexander Neumann
36fa1f8c20
Merge pull request #1894 from restic/fix1893
...
Return error when exclude file cannot be read
2018-07-22 14:34:27 +02:00
Alexander Neumann
445fb23b6d
Rework issue templates for Bug reports and Features
2018-07-22 14:26:23 +02:00
Alexander Neumann
5f79b4cb6c
Update issue template again
2018-07-22 14:21:08 +02:00
Alexander Neumann
8e15b59347
Use underline style markup for issue/PR templates
2018-07-22 14:17:53 +02:00
Alexander Neumann
6e2e957332
Add entry to changelog
2018-07-22 14:16:08 +02:00
Alexander Neumann
7ffc03ff8f
Update github.com/kurin/blazer to 0.5.0
...
This includes support for the upcoming B2 application keys feature.
2018-07-22 14:12:02 +02:00
Alexander Neumann
44924ba043
restorer: Fix traverseTree
...
traverseTree() was meant to call enterDir() whenever a directory is
selected for restore, either explicitly or implicitly (=contains a file
which is to be restored). After restoring a file, leaveDir() is called
in reverse order for all intermediate directories so that the metadata
can be restored.
When a directory is selected implicitly, the metadata for it is
restored. This is different from the previous restorer behavior, which
created implicitly selected intermediate directories with permissions
0700 (only user can read/write it).
This commit changes the behavior back to the old one. Only a directory
is explicitly selected for restore, enterDir()/leaveDir() are called for
it. Otherwise, only visitNode() is called, so visitNode() needs to make
sure the parent directory exists. If the directory is explicitly
included, leaveDir() will then restore the metadata correctly.
When we decide to change the behavior (restore metadata for all
intermediate directories, even if selected implicitly), we should do
that in the selection functions, not here.
This finally resolves #1870
2018-07-21 23:24:40 +02:00
Alexander Neumann
ce19f26948
restorer: Add tests for traverseTree
2018-07-21 23:24:40 +02:00
Alexander Neumann
74016d5981
restorer: Fix return of saveSnapshot
2018-07-21 23:24:40 +02:00
Alexander Neumann
57636a4573
restorer: Run tests in the same package
2018-07-21 23:24:40 +02:00
Alexander Neumann
4f6d2502f7
restorer: Add test for restore with include filter
2018-07-21 23:24:40 +02:00
Garry McNulty
f1f69bc648
check: Use --cache-dir argument
...
Closes #1880
2018-07-20 20:51:20 +01:00
Alexander Neumann
d7551d7b0c
Add entry to changelog
2018-07-18 21:41:20 +02:00
Alexander Neumann
fb74de6360
Return an error when exclude files cannot be read
2018-07-18 21:39:07 +02:00
Alexander Neumann
67535e00a8
Merge pull request #1889 from ProactiveServices/patch-3
...
doc: Minor grammar, RST syntax fixes
2018-07-18 21:22:10 +02:00
Alexander Neumann
19592285eb
Merge pull request #1888 from ProactiveServices/patch-2
...
doc: Minor grammar fixes
2018-07-18 21:21:52 +02:00
Alexander Neumann
f64862722a
Merge pull request #1887 from restic/disable-error-size
...
checker: Disable size check for now
2018-07-18 21:19:54 +02:00
Adam Piggott
254239c2a9
doc: Minor grammar, RST syntax fixes
...
Fix unescaped backslash
Fix wording of Windows installation
2018-07-18 02:28:23 +01:00
Adam Piggott
cce1a1f768
doc: Minor grammar fixes
2018-07-18 02:25:31 +01:00
Alexander Neumann
754482fe6c
checker: Disable size check for now
2018-07-15 21:52:38 +02:00
Alexander Neumann
73153dbd3f
Merge pull request #1885 from restic/create-restore-target
...
restore: Make sure the target directory exists
2018-07-15 16:28:25 +02:00
Alexander Neumann
92421ec47f
restore: Make sure target directory exists
2018-07-15 16:02:04 +02:00
Alexander Neumann
9acc9243ba
Add test for not-existing top-level dir and top-level file
2018-07-15 16:00:26 +02:00
Alexander Neumann
df64998649
Merge pull request #1882 from duzvik/aws-credentials-priority
...
Change AWS credentials priority, to accept AWS_SESSION_TOKEN
2018-07-14 20:48:42 +02:00
Alexander Neumann
64d27eed86
doc: Improve dump to stdout
...
Closes #1884
2018-07-14 20:45:52 +02:00
Alexander Neumann
abb18a830c
Fix test
2018-07-14 11:51:34 +02:00
denis.uzvik
1e42f4f300
S3 backend: accept AWS_SESSION_TOKEN
2018-07-12 16:18:19 +03:00
Alexander Neumann
bd742ddb69
cache: Don't recreate CACHEDIR.TAG
2018-07-08 12:05:12 +02:00
Alexander Neumann
b511f4dce2
Improve help message for check
2018-07-05 22:19:08 +02:00
Alexander Neumann
7961740dcc
Fix link
2018-07-05 21:03:40 +02:00
Alexander Neumann
dc3032c360
Mention that AppsCode is sponsoring backend tests
2018-07-05 21:01:57 +02:00
Alexander Neumann
44fb2a860f
Merge pull request #1861 from McKael/fix_find_ignore_case
...
Fix find -i (case-insensitive search)
2018-06-24 21:08:03 +02:00
Mikael Berthe
fbf8073dfc
Fix find -i (case-insensitive search)
2018-06-24 19:35:11 +02:00
Alexander Neumann
7ddf91b65c
Merge pull request #1772 from ifedorenko/restore-verify
...
Restore verify
2018-06-23 14:50:29 +02:00
Alexander Neumann
8dae2de2ce
Merge pull request #1858 from restic/fix-1857
...
Allow excluding files with $ via --exclude-file
2018-06-22 21:22:23 +02:00
Alexander Neumann
03a0377410
Merge pull request #1856 from stkw0/master
...
Ignore ExcludeOtherFs if Stdin is true
2018-06-22 20:49:34 +02:00
Alexander Neumann
025ec9dff5
Allow excluding files with $ via --exclude-file
...
Previously it wasn't possbile to exclude files with a literal dollar
sign (`$`) via exclude files, now users can write `$$` for that.
2018-06-22 20:46:04 +02:00
David Roman
2384c1cee7
Ignore ExcludeOtherFs if Stdin is true
...
Closes : #1807
2018-06-21 23:12:38 +02:00
Alexander Neumann
bb2ad76833
Merge pull request #1855 from restic/fix-1854
...
Allows saving files/dirs on different file systems together with `--one-file-system`.
2018-06-21 20:48:33 +02:00
Alexander Neumann
30cfd13328
Add changelog
2018-06-20 23:05:09 +02:00
Alexander Neumann
9ffc26883a
archiver: Unroll tree
2018-06-20 22:56:41 +02:00
Alexander Neumann
83c51db903
fs: Add helper functions ReadDir/ReadDirNames
2018-06-20 22:53:53 +02:00
Alexander Neumann
d30d5d4473
Merge pull request #1853 from skriss/list-keys-json-output
...
add JSON output support for restic key list cmd
2018-06-19 21:28:45 +02:00
Steve Kriss
5088905502
add JSON output support for restic key list cmd
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-06-18 15:26:26 -07:00
Alexander Neumann
ae72b438b0
doc: Fix typo
2018-06-18 22:19:01 +02:00
Alexander Neumann
ddf2065ce2
doc: Split references out into smaller files
...
Closes #1852
2018-06-18 22:17:48 +02:00
Alexander Neumann
228a970540
doc: Add paragraph about finding path in file
2018-06-17 20:49:11 +02:00
Alexander Neumann
c7a8086c19
Merge pull request #1851 from restic/fix-1849
...
walker: Don't ignore empty trees by default
2018-06-17 12:25:33 +02:00
Alexander Neumann
c2c06ae2c9
walker: Don't ignore empty trees by default
...
Closes #1849
2018-06-17 09:49:03 +02:00
Alexander Neumann
1824168aa3
Merge pull request #1848 from iquiw/netbsd-support
...
Support NetBSD without fuse
2018-06-16 17:12:20 +02:00
Iku Iwasa
350761f1ba
Add cross-compilation for netbsd/386 and netbsd/amd64
2018-06-16 21:05:26 +09:00
Alexander Neumann
3231945a85
Merge pull request #1846 from cbergmann/patch-1
...
Clarify live status display
2018-06-16 11:19:58 +02:00
Iku Iwasa
f080142137
Support NetBSD without fuse
2018-06-16 15:55:04 +09:00
Clemens Bergmann
ff785924de
Clarify live status display
...
When I backup one of my filesystems which has a lot of Hard Links (Backup directory of burp) the live status shows me 4.5 TB but it only takes up 1.2 TB of space in the repository. This is confusing because my repo is on S3 and I feared a huge Bill. This change should clarify this.
2018-06-15 12:41:48 +02:00
Alexander Neumann
393a7266c9
Merge pull request #1845 from dimejo/patch-2
...
doc: enhance FAQ about AV software on Windows
2018-06-14 20:48:15 +02:00
Alex JOST
cb8d2d3df5
doc: enhance FAQ about AV software on Windows
...
As discussed in #1840 , realtime protection of AV software can cause serious performance degradation on Windows OS.
2018-06-14 19:39:04 +02:00
Alexander Neumann
a884ce1566
Merge pull request #1841 from jernst/master
...
Added documentation about what happens if you run out of space during a backup
2018-06-14 19:13:31 +02:00
Alexander Neumann
5ae8316c24
Merge pull request #1844 from gsanchietti/patch-1
...
Documentation: minor RST syntax fixes
2018-06-14 19:12:57 +02:00
Giacomo Sanchietti
85eca1b5e9
Documentation: minor RST syntax fixes
...
- Fix bad link syntax
- Fix compilation warning
2018-06-14 15:01:49 +02:00
Johannes Ernst
a1536f38fa
Fixed typo and grammar per feedback on PR.
2018-06-13 00:24:42 +00:00
Johannes Ernst
888f52afd1
Added documentation about what happens if you run out of space
...
during a backup, based on conversation on the forum:
https://forum.restic.net/t/limited-backup-destination-space/733/6
2018-06-11 21:20:08 +00:00
Igor Fedorenko
e206680947
restore: New --verify flag to verify restored files content
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Igor Fedorenko
5fa6dc53cb
Refactor: introduced restorer tree visitor
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Igor Fedorenko
26be094f28
Refactor: moved restorer to separate package
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-06-11 07:57:22 -04:00
Alexander Neumann
e4c0d77bdd
Add VERSION for 0.9.1
2018-06-10 11:31:03 +02:00
Alexander Neumann
1dd655dad2
Generate CHANGELOG.md for 0.9.1
2018-06-10 11:30:53 +02:00
Alexander Neumann
581d0984fe
Fix changelog entry
2018-06-10 11:29:57 +02:00
Alexander Neumann
e62add84bc
Move changelog files for 0.9.1
2018-06-10 11:22:32 +02:00
Alexander Neumann
63779c1eb4
Merge pull request #1839 from restic/fix-find
...
Fix find, do not skip some snapshots
2018-06-10 10:08:47 +02:00
Alexander Neumann
c204382ea9
Revert "Fix integration tests on Windows"
...
This reverts commit 33dbd0ba5c
.
2018-06-10 00:01:28 +02:00
Alexander Neumann
321efec60c
Fix integration tests on Windows
2018-06-10 00:00:22 +02:00
Alexander Neumann
33dbd0ba5c
Fix integration tests on Windows
2018-06-09 23:58:44 +02:00
Alexander Neumann
9a73869c27
Update docs for RHEL/CentOS
2018-06-09 23:41:40 +02:00
Alexander Neumann
8f26fe271c
ls: Use walker for ls
2018-06-09 23:35:20 +02:00
Alexander Neumann
251335f124
Add entry to changelog
2018-06-09 23:35:20 +02:00
Alexander Neumann
081743d0a5
find: Use walker.Walk
2018-06-09 23:35:20 +02:00
Alexander Neumann
3a86f4852b
Add walker for trees in the repo
2018-06-09 23:35:20 +02:00
Alexander Neumann
14aead94b3
filter: Allow double wildcard in ChildMatch
2018-06-09 23:18:13 +02:00
Alexander Neumann
ce01ca30d6
find: Correct tree pruning optimization
...
The `find` command will now take care to only mark trees as "not found"
when the pattern couldn't be found within any subtree.
Closes #1825 , #1823
2018-06-09 18:59:13 +02:00
Alexander Neumann
e2d347a698
find: Use OS independent slash-based format
2018-06-09 18:58:13 +02:00
Alexander Neumann
42ebb0a0a6
backup: Parse timestamp earlier
2018-06-09 18:21:12 +02:00
Alexander Neumann
419acad3c3
Merge pull request #1837 from restic/fix-1833
...
cache: Ensure failed downloads are retried
2018-06-09 18:20:21 +02:00
Alexander Neumann
810b5ea076
Add entry to changelog
2018-06-09 17:55:51 +02:00
Alexander Neumann
fc5439a37a
cache: Ensure failed downloads are retried
...
This fixes #1833 , which consists of two different bugs:
* The `defer` in `cacheFile()` may remove a channel from the
`inProgress` map although it is not responsible for downloading the
file
* If the download fails, goroutines waiting for the file to be cached
assumed that the file was there, there was no way to signal the
error.
2018-06-09 17:50:56 +02:00
Alexander Neumann
48aab8bd65
Merge pull request #1836 from restic/update-blazer
...
Update github.com/kurin/blazer
2018-06-09 14:31:12 +02:00
Alexander Neumann
6fbcd1694b
Add entry to changelog
2018-06-09 14:31:02 +02:00
Alexander Neumann
494fe2a8b5
Merge pull request #1835 from restic/fix-1834
...
backup: Fix deadlock
2018-06-09 14:28:16 +02:00
Alexander Neumann
f761068f4e
Update github.com/kurin/blazer
2018-06-09 12:32:18 +02:00
Alexander Neumann
c44e808aa5
backup: Fix deadlock
...
When the archiver is faster than the scanner, restic deadlocks. This
commit adds a `finished` channel to the struct in `ui/backup.go` so that
scanner results are ignored when the archiver is already finished.
Closes #1834
2018-06-09 12:15:19 +02:00
Alexander Neumann
ab37c6095a
Merge pull request #1821 from michaelkoetter/fix-1795
...
#1795 use unix.IoctlGetWinsize to get terminal size
2018-06-07 20:20:06 +02:00
Michael Kötter
d6fd94e49d
Don't run Solaris build for go1.9
2018-06-04 15:04:50 +02:00
Michael Kötter
53040a2e34
add "solaris/amd64" to cross-compile archs
2018-06-04 12:51:34 +02:00
Alexander Neumann
cfc19b4582
Merge pull request #1828 from restic/handle-s3-list-errors
...
s3: Pass list errors up to the caller
2018-06-02 10:34:49 +02:00
Alexander Neumann
141fabdd09
s3: Pass list errors up to the caller
2018-06-01 22:15:23 +02:00
Alexander Neumann
d49ca42771
Merge pull request #1827 from restic/azure-large-files
...
azure: Support uploading large files
2018-06-01 18:37:26 +02:00
Alexander Neumann
f6fded729d
Add entry to changelog
2018-06-01 14:52:16 +02:00
Alexander Neumann
465700595c
azure: Support uploading large files
...
Closes #1822
2018-06-01 14:52:16 +02:00
Alexander Neumann
0fcd9d6926
Merge pull request #1824 from rfjakob/ssh_command_exited
...
sftp: persist "ssh command exited" error
2018-05-31 21:26:39 +02:00
Jakob Unterwurzacher
dd3b9910ee
sftp: persist "ssh command exited" error
...
If our ssh process has died, not only the next, but all subsequent
calls to clientError() should indicate the error.
restic output when the ssh process is killed with "kill -9":
Save(<data/afb68adbf9>) returned error, retrying after 253.661803ms: Write: failed to send packet header: write |1: file already closed
Save(<data/afb68adbf9>) returned error, retrying after 580.752212ms: ssh command exited: signal: killed
Save(<data/afb68adbf9>) returned error, retrying after 790.150468ms: ssh command exited: signal: killed
Save(<data/afb68adbf9>) returned error, retrying after 1.769595051s: ssh command exited: signal: killed
[...]
error in cleanup handler: ssh command exited: signal: killed
Before this patch:
Save(<data/de698d934f>) returned error, retrying after 252.84163ms: Write: failed to send packet header: write |1: file already closed
Save(<data/de698d934f>) returned error, retrying after 660.236963ms: OpenFile: failed to send packet header: write |1: file already closed
Save(<data/de698d934f>) returned error, retrying after 568.049909ms: OpenFile: failed to send packet header: write |1: file already closed
Save(<data/de698d934f>) returned error, retrying after 2.428813824s: OpenFile: failed to send packet header: write |1: file already closed
[...]
error in cleanup handler: failed to send packet header: write |1: file already closed
2018-05-30 19:28:14 +02:00
Alexander Neumann
185b60c22b
Document project governance
2018-05-28 22:29:06 +02:00
Michael Kötter
589c23dc23
#1795 use unix.IoctlGetWinsize to get terminal size
2018-05-27 23:44:48 +02:00
Alexander Neumann
0183fea926
Merge pull request #1820 from restic/fix-1803
...
termstatus: Fix panic for non-terminal runs
2018-05-27 13:08:25 +02:00
Alexander Neumann
7d9642523b
termstatus: Fix panic for non-terminal runs
...
Closes #1803
2018-05-27 12:52:01 +02:00
Alexander Neumann
4bf07a74a0
Merge pull request #1806 from mholt/patch-1
...
doc: Clarify multiple forget policies get ORed
2018-05-26 11:03:22 +02:00
Alexander Neumann
2a976d795f
b2: Remove extra error check
2018-05-26 10:12:30 +02:00
Alexander Neumann
1892b314f8
Merge pull request #1815 from restic/update-blazer
...
Update github.com/kurin/blazer
2018-05-25 20:46:35 +02:00
Alexander Neumann
b7bed406b9
Update github.com/kurin/blazer
2018-05-25 20:26:26 +02:00
Matt Holt
ee4202f7c3
doc: Clarify multiple forget policies get ORed
2018-05-23 17:28:02 -06:00
Alexander Neumann
4cd28713b6
Merge pull request #1802 from restic/rclone-add-limits
...
rclone: Add limiting bandwidth to the rclone backend
2018-05-22 21:19:52 +02:00
Alexander Neumann
e3fe87f269
Remove superseded feature from the CHANGELOG
...
For a discussion please see https://github.com/restic/restic/issues/1796
2018-05-22 20:57:07 +02:00
Alexander Neumann
a02698fcdd
Add entry to changelog
2018-05-22 20:48:29 +02:00
Alexander Neumann
bfd923e81e
rclone: Respect bandwith limits
2018-05-22 20:48:17 +02:00
Alexander Neumann
20bfed5985
Update build.go
2018-05-21 20:31:19 +02:00
Alexander Neumann
e40191942d
Add VERSION for 0.9.0
2018-05-21 15:57:32 +02:00
Alexander Neumann
abd34ab76f
Update manpages and auto-completion
2018-05-21 15:57:32 +02:00
Alexander Neumann
4b43a269ee
Generate CHANGELOG.md for 0.9.0
2018-05-21 15:57:22 +02:00
Alexander Neumann
e2b7dc6528
Move changelog files for the 0.9.0 release
2018-05-21 15:57:02 +02:00
Alexander Neumann
d2431b667f
Merge pull request #1794 from restic/fix-1789
...
Update docs, clarify --cacert
2018-05-21 12:35:06 +02:00
Alexander Neumann
b70fdf61c4
Merge pull request #1791 from restic/fix-1790
...
archiver: Fix backup from stdin
2018-05-21 12:07:55 +02:00
Alexander Neumann
e6f25c4811
Update docs, clarify --cacert
...
Closes #1789
2018-05-21 12:06:30 +02:00
Alexander Neumann
adb682bc43
archiver: Don't open files with O_NONBLOCK
...
This is not necessary any more, we're doing an lstat() before opening
an item, so we already known it's a file and not a pipe.
2018-05-20 16:11:51 +02:00
Alexander Neumann
1e9744c9a4
archiver: Refuse to save an empty snapshot
2018-05-20 16:11:51 +02:00
Alexander Neumann
9a02f17cc2
archiver: Add tests for Save() for fs.Reader
2018-05-20 16:11:36 +02:00
Alexander Neumann
c284712cae
Merge pull request #1784 from restic/improve-error-list
...
list: Improve error message
2018-05-18 21:53:09 +02:00
Alexander Neumann
2dbdf381b2
Merge pull request #1782 from skriss/add-s3-file-creds
...
Add S3 file creds and reorder creds chain
2018-05-18 21:52:54 +02:00
Alexander Neumann
a1a49ce211
Merge pull request #1787 from tyll/patch-1
...
Use more descriptive long option
2018-05-18 21:51:24 +02:00
Till Maas
3252e4200c
Use more descriptive long option
...
-r could also mean recursive so use --repo to be clear.
2018-05-18 19:25:56 +02:00
Alexander Neumann
8d9d218d1c
list: Improve error message
...
Before:
$ restic list
Fatal: type not specified
After:
$ restic list
Fatal: type not specified, usage: list [blobs|packs|index|snapshots|keys|locks]
Closes #1783
2018-05-17 19:41:56 +02:00
Steve Kriss
0785fbd418
changelog entry
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-17 10:09:38 -07:00
Steve Kriss
b358dd369b
S3: rearrange credentials chain to be standard
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-16 16:49:33 -07:00
Steve Kriss
d67b9a32c6
S3: add file credentials to chain
...
Signed-off-by: Steve Kriss <steve@heptio.com>
2018-05-16 16:35:14 -07:00
Alexander Neumann
ecfe59235e
Merge pull request #1781 from McKael/fix_restick_backup_rejectByDevice
...
Fix restic backup --one-file-system /
2018-05-16 21:58:58 +02:00
Mikael Berthe
a868a30f4d
Fix restic backup --one-file-system /
...
This patch should fix the following panic when trying to backup the
root filesystem with thre --one-file-system flag:
% restic backup --one-file-system /
(...)
panic: item /, device id 2082 not found, allowedDevs: map[/:2082]
2018-05-16 13:44:14 +02:00
Alexander Neumann
347a645450
Fix double error message
2018-05-15 11:03:33 +02:00
Alexander Neumann
9f5565b0fc
Merge pull request #1735 from mholt/forget-max-age
...
forget: Add --max-age policy to set hard cutoff for removing snapshots
2018-05-14 21:18:06 +02:00
Alexander Neumann
fd979ab4c5
Merge pull request #1779 from restic/improve-error
...
Improve error message for readdir/readdirnames
2018-05-14 21:17:57 +02:00
Alexander Neumann
375868edcf
Add documentation
2018-05-13 12:54:23 +02:00
Alexander Neumann
060d8b57e0
Restructure TestApplyPolicy
2018-05-13 12:48:42 +02:00
Alexander Neumann
cc627e832b
Add custom Duration type
2018-05-13 12:48:42 +02:00
Alexander Neumann
5a0f0e3faa
Add support for keeping a range of snapshots
2018-05-13 12:48:42 +02:00
Matthew Holt
b52f2aa9a4
forget: Add policy to keep snapshots before a date
2018-05-13 12:48:10 +02:00
Alexander Neumann
60ea2435be
Improve error message for readdir/readdirnames
...
As mentioned in the forum[1], restic does not include the dir name when
readdir/readdirnames fails.
[1] https://forum.restic.net/t/readdirnames-readdirent-no-such-file-or-directory/653
2018-05-13 10:34:50 +02:00
Alexander Neumann
159badf5ba
Merge pull request #1778 from restic/fix-1771
...
archiver: Improve error handling
2018-05-13 00:13:54 +02:00
Alexander Neumann
903a3a31dc
Merge pull request #1776 from restic/fix-1775
...
Always use cleaned path for excludes
2018-05-13 00:13:52 +02:00
Alexander Neumann
548227e6df
Merge pull request #1773 from ExoUNX/patch-1
...
Fix small typo
2018-05-13 00:13:49 +02:00
Alexander Neumann
cd03275005
Merge pull request #1774 from ExoUNX/patch-2
...
Fix formatting
2018-05-13 00:13:46 +02:00
Alexander Neumann
e43c9202a6
archiver: Make sure backend error is passed up
2018-05-12 23:55:59 +02:00
Alexander Neumann
c5e75d1c98
archiver: Add test for early abort on unhandled error
2018-05-12 23:55:59 +02:00
Alexander Neumann
526956af35
archiver: Read files/dirs in order
2018-05-12 23:55:54 +02:00
Alexander Neumann
256104111d
archiver: Clarify names
2018-05-12 23:55:54 +02:00
Alexander Neumann
21c83b1725
archiver: Add high-level documentation
2018-05-12 23:55:54 +02:00
Alexander Neumann
581c62ee72
archiver: Improve error handling
...
This commit changes how the worker goroutines for saving e.g. blobs
interact. Before, it was possible to get stuck sending an instruction to
archive a file or dir when no worker goroutines were available any more.
This commit introduces a `done` channel for each of the worker pools,
which is set to the channel returned by `tomb.Dying()`, so it is closed
when the first worker returned an error.
2018-05-12 23:55:54 +02:00
Alexander Neumann
ef7747313d
backup: Use absolute paths for allowed devices
2018-05-11 21:32:44 +02:00
Alexander Neumann
18d4ac2fd9
backup: Always use cleaned path for excludes
2018-05-11 21:32:41 +02:00
Alexander Neumann
9180e2c48a
Remove unneeded file excludes
2018-05-11 21:26:10 +02:00
Gaige Lama
a63989afcd
Ran gofmt on backend.go
...
I ran gofmt on backend.go, this appears to valid edit.
2018-05-11 11:07:16 -04:00
Gaige Lama
d3c0bd6d0e
Fix small typo
...
"explicitely" should be "explicitly"
2018-05-11 10:55:12 -04:00
Alexander Neumann
fcfa6f0355
build: Add option to enable PIE build mode
2018-05-11 09:50:10 +02:00
Alexander Neumann
580f90d745
Merge pull request #1770 from restic/handle-blob-save-errors
...
archiver: Correct error handling
2018-05-10 22:36:00 +02:00
Alexander Neumann
c7b624ba0d
Merge pull request #1769 from McKael/cat_snapshot_stacktrace
...
cat snapshot: Do not display a stack trace with invalid IDs
2018-05-10 21:41:04 +02:00
Alexander Neumann
ca4af43c03
archiver: Return low-level errors
...
This commit changes the archiver so that low-level errors saving data to
the repo are returned to the caller (instead of being handled by the
error callback function). This correctly bubbles up errors like a full
temp file system and makes restic abort early and makes all other worker
goroutines exit.
2018-05-10 21:30:09 +02:00
Alexander Neumann
1f2463f42e
archiver: Return correct error
2018-05-10 20:48:00 +02:00
Alexander Neumann
157c854d04
backup: Remove errored files from status display
2018-05-10 20:48:00 +02:00
Mikael Berthe
ffc276a603
cat: Do not display a stack trace
...
Don't show a stack trace when "restic cat snapshot" is invoked with
invalid/nonexistent IDs.
2018-05-10 19:22:46 +02:00
Alexander Neumann
e42b7db008
Merge pull request #1767 from restic/update-blazer
...
Update github.com/kurin/blazer
2018-05-08 22:48:54 +02:00
Alexander Neumann
024148cac9
Update github.com/kurin/blazer
2018-05-08 22:33:55 +02:00
Alexander Neumann
8943741a0b
Fix code block
2018-05-08 20:36:58 +02:00
Alexander Neumann
95c5517c35
Merge pull request #1765 from restic/update-deps
...
Update golang.org/x/{sys,crypto}
2018-05-08 19:34:30 +02:00
Alexander Neumann
06179a7e81
Change URL for Travis
2018-05-08 19:13:39 +02:00
Alexander Neumann
cf1fb50f9c
Merge pull request #1764 from restic/fix-display
...
Fix display
2018-05-07 23:30:50 +02:00
Alexander Neumann
6793300850
Raise minimum Go version to 1.9
...
The golang.org/sys/crypto library uses math/bits, which was only
introduced in Go 1.9.
2018-05-07 23:28:48 +02:00
Alexander Neumann
2cbdfbf652
Update golang.org/x/{sys,crypto}
2018-05-07 22:47:39 +02:00
Alexander Neumann
b2208bb9c2
Rework termstatus
...
This now keeps the cursor at the first column of the first status line
so that messages printed to stdout or stderr by some other part of the
progarm will still be visible. The message will overwrite the status
lines, but those are easily reprinted on the next status update.
2018-05-07 21:40:07 +02:00
Alexander Neumann
4c25495d68
backup: Hide percent until total size is known
2018-05-06 20:20:25 +02:00
Alexander Neumann
abdd59ea1b
Merge pull request #1757 from restic/fix-debug-1755
...
Fix panic for nil-pointer dereference
2018-05-03 21:11:37 +02:00
Alexander Neumann
05ca903d48
Mention that first backup will take longer
2018-05-03 20:57:30 +02:00
Alexander Neumann
fd77646f8b
Fix panic for nil-pointer dereference
...
Closes #1755
2018-05-03 20:49:30 +02:00
Alexander Neumann
2a67258867
Merge pull request #1754 from restic/fix-quiet
...
backup: Disable status output for --quiet
2018-05-02 21:42:17 +02:00
Alexander Neumann
fca4fe4459
backup: Disable status output for --quiet
2018-05-02 21:24:18 +02:00
Alexander Neumann
26757ae2e5
Merge pull request #1750 from restic/archiver-fix-small-things
...
backup: Tune user interface a bit
2018-05-01 23:15:54 +02:00
Alexander Neumann
9d6890a236
Merge pull request #1751 from restic/fix-archiver
...
archiver: Use lstat before open/fstat
2018-05-01 23:15:32 +02:00
Alexander Neumann
2218ecd049
archiver: Use lstat before open/fstat
...
The previous code tried to be as efficient as possible and only do a
single open() on an item to save, and then fstat() on the fd to find out
what the item is (file, dir, other). For normal files, it would then
start reading the data without opening the file again, so it could not
be exchanged for e.g. a symlink.
This behavior starts the watchdog on my machine when /dev is saved
with restic, and after a few seconds, the machine reboots.
This commit reverts the behavior to the strategy the old archiver code
used: run lstat(), then decide what to do. For normal files, open the
file and then run fstat() on the fd to verify it's still a normal file,
then start reading the data.
The downside is that for normal files we now do two stat() calls
(lstat+fstat) instead of only one. On the upside, this does not start
the watchdog. :)
2018-05-01 23:05:50 +02:00
Alexander Neumann
d0974c155d
backup: Tune user interface a bit
...
Make non-verbose mode a bit more verbose.
2018-05-01 22:18:32 +02:00
Alexander Neumann
8026e6fdfb
Merge pull request #1749 from restic/add-cache-cmd
...
Add 'cache' command to list and cleanup cache dirs
2018-05-01 21:25:52 +02:00
Alexander Neumann
01f9662614
Merge pull request #1748 from restic/detect-bom
...
Respect Encoding and Byte Order Mark when reading text files
2018-05-01 21:25:46 +02:00
Alexander Neumann
f928aeec34
Add 'cache' command
2018-05-01 16:27:44 +02:00
Alexander Neumann
f77bc0fae8
Use textfile.Read() to read files
...
This converts the text to UTF-8 and removes a Byte Order Mark.
2018-05-01 16:25:53 +02:00
Alexander Neumann
eb6650b201
Add textfile
2018-05-01 14:38:59 +02:00
Alexander Neumann
bc68d55e94
fs: Add TestTempFile
2018-05-01 14:38:41 +02:00
Alexander Neumann
ecbbd851a1
Merge pull request #1744 from restic/fix-parent-detect
...
backup: Clean target paths before finding parent
2018-05-01 11:13:19 +02:00
Alexander Neumann
336719b058
Merge pull request #1746 from brycied00d/fix_tls-client-cert_argument_parsing
...
http backend: Parse the correct argument when loading --tls-client-cert
2018-05-01 10:22:10 +02:00
Bryce Chidester
e9f1721678
http backend: Parse the correct argument when loading --tls-client-cert
...
Previously, the function read from ARGV[1] (hardcoded) rather than the
value passed to it, the command-line argument as it exists in globalOptions.
Resolves #1745
2018-04-30 15:21:09 -07:00
Alexander Neumann
64d98945a6
backup: Clean target paths before finding parent
...
This resolves an issue described in the forum where restic could not
find a parent snapshot if the target path ends with a slash:
https://forum.restic.net/t/new-archiver-code-please-test/623/23
2018-04-30 22:03:11 +02:00
Alexander Neumann
84f82dae1a
Correct typo
...
Closes #1743
2018-04-30 21:41:30 +02:00
Alexander Neumann
6bfd9f833b
Merge pull request #1742 from tbm/typos
...
Fix typos in section on rclone
2018-04-30 19:21:19 +02:00
Martin Michlmayr
bb1a22d1e6
Fix typos in section on rclone
2018-04-30 18:32:10 +02:00
Alexander Neumann
438719f269
Merge pull request #1741 from restic/improve-archiver
...
Improve archiver, fix hang on fifos
2018-04-30 16:00:10 +02:00
Alexander Neumann
c83c03ed63
archiver: Fix blocking on pipes
2018-04-30 15:34:58 +02:00
Alexander Neumann
19b9c881ca
fs: Add O_NONBLOCK
2018-04-30 15:34:58 +02:00
Alexander Neumann
4e34325035
archiver: Process dirs concurrently
2018-04-30 15:13:28 +02:00
Alexander Neumann
78bd591c7c
archiver: Improve buffer pool
2018-04-30 15:13:28 +02:00
Alexander Neumann
39ac12f6ea
archiver: Correct comment
2018-04-30 14:19:07 +02:00
Alexander Neumann
400730afca
archiver: Improve memory usage, tune buffer pool
2018-04-30 14:19:07 +02:00
Alexander Neumann
d80e108b03
backup: Clear status lines on finish
2018-04-30 14:19:07 +02:00
Alexander Neumann
846c2b6869
backup: Fix ETA calculation for >= 100%
2018-04-30 14:19:07 +02:00
Alexander Neumann
d8bbe5dc84
Print repository ID after opening
2018-04-30 14:19:07 +02:00
Alexander Neumann
d926b9fd80
Add profile build flag
2018-04-30 14:19:04 +02:00
Alexander Neumann
4ba8d40282
Add block profile option
2018-04-30 14:18:45 +02:00
Alexander Neumann
4fb1401266
Fix --cacert help text
2018-04-29 13:17:33 +02:00
Alexander Neumann
6d4c40f8d0
Add section about file read concurrency
2018-04-28 22:39:56 +02:00
Alexander Neumann
56e394ac33
Merge pull request #1494 from restic/new-archiver
...
New archiver code
2018-04-28 22:24:39 +02:00
Alexander Neumann
c3cc5d7cee
Update docs
2018-04-28 22:08:11 +02:00
Alexander Neumann
6b12b92339
Add entry to changelog
2018-04-28 22:08:11 +02:00
Alexander Neumann
16c314ab7f
termstatus: Don't print status if in background
2018-04-28 22:08:11 +02:00
Alexander Neumann
1449d7dc29
Remove background checking code
2018-04-28 22:08:11 +02:00
Alexander Neumann
0e78ac92d8
Use new archiver code for backup
2018-04-28 22:08:11 +02:00
Alexander Neumann
c703d21d55
Vendor gopkg.in/tomb.v2
2018-04-27 21:42:15 +02:00
Alexander Neumann
1af96fc6dd
Add termstatus
2018-04-27 21:42:15 +02:00
Alexander Neumann
9fac2ca832
Add flags to set verbosity
2018-04-25 14:42:45 +02:00
Alexander Neumann
a5c0cf2324
Add workaround for symlinked temp dir on darwin
...
Chdir to the tempdir, then use os.Getwd() to get the name that
filepath.Abs() uses (and stores in the Snapshot).
2018-04-25 14:42:45 +02:00
Alexander Neumann
38926d8576
Use new archiver code in tests
2018-04-25 14:42:45 +02:00
Alexander Neumann
f279731168
Add new archiver code
2018-04-25 14:42:45 +02:00
Alexander Neumann
76b616451f
Remove unneeded code
2018-04-23 21:40:33 +02:00
Alexander Neumann
fd12a3af20
Remove old archiver code
2018-04-23 21:40:33 +02:00
Alexander Neumann
3cd92efdcf
Vendor github.com/mattn/go-isatty
2018-04-22 11:37:05 +02:00
Alexander Neumann
b804279fe8
integration tests: Don't print anything to stdout
2018-04-22 11:37:05 +02:00
Alexander Neumann
a56b8fad87
repository: Improve buffer pooling
2018-04-22 11:37:05 +02:00
Alexander Neumann
4c00efd4bf
Vendor go-cmp
2018-04-22 11:37:05 +02:00
Alexander Neumann
b6f98bdb02
node: Fill minimal info
2018-04-22 11:37:05 +02:00
Alexander Neumann
c4b2486b7c
fs: Add interface and FS implementations
...
This adds two implementations of the new `FS` interface: One for the local
file system (`Local`) and one for a single file read from an
`io.Reader` (`Reader`).
2018-04-22 11:37:05 +02:00
Alexander Neumann
83ca08245b
checker: Check metadata size and blob sizes
2018-04-22 11:37:05 +02:00
Alexander Neumann
a069467e72
ls: Improve output
2018-04-22 11:37:05 +02:00
Alexander Neumann
6a7c23d2ae
tree: Add convenience functions
2018-04-22 11:37:05 +02:00
Alexander Neumann
cc847a3d6d
tree: Improve error for pre-existing node
2018-04-22 11:37:05 +02:00
Alexander Neumann
baebf45e2e
FindLatestSnapshot: Use absolute paths
2018-04-22 11:37:05 +02:00
Alexander Neumann
fa4f438bc1
snapshot: Do not modify slice of paths
2018-04-22 11:37:05 +02:00
Alexander Neumann
4e0b2a8e3a
snapshot: correct error handling for filepath.Abs
2018-04-22 11:37:05 +02:00
Alexander Neumann
0532f08048
Add test.Helper, also works with Go 1.8
2018-04-22 11:37:05 +02:00
Alexander Neumann
a472868e06
fs: Add TestChdir()
2018-04-22 11:37:05 +02:00
Alexander Neumann
e4fdc5eb76
fs: Add IsRegularFile()
2018-04-22 11:37:05 +02:00
Alexander Neumann
09365cc4ea
Add --trace-profile
2018-04-22 11:37:05 +02:00
Eri Bastos
2aa6b49651
Return exit code 130 when SIGINT is received
2018-04-20 21:09:50 +02:00
Alexander Neumann
7877797c7e
Merge pull request #1720 from mholt/new-password-flag
...
key: Add --new-password flag for non-interactive password changes
2018-04-20 14:52:41 +02:00
Matthew Holt
1a26355dbe
Add changelog file
2018-04-20 14:29:39 +02:00
Matthew Holt
c5829e9ffc
key: Add flag for non-interactive password changes
2018-04-20 14:29:39 +02:00
Alexander Neumann
b5b246edd5
Add entry to changelog
2018-04-20 14:23:37 +02:00
Alexander Neumann
ee5e14d536
Merge pull request #1731 from restic/fix-1730
...
Do not restore sockets, correct error handling
2018-04-20 14:21:01 +02:00
Alexander Neumann
09bd924710
Do not restore sockets, correct error handling
...
Closes #1730
2018-04-20 13:53:11 +02:00
Alexander Neumann
a9c2e84ccd
Merge pull request #1718 from ardichoke/patch-1
...
Update Autocompletion Generation Documentation
2018-04-13 22:23:37 +02:00
Ryan DeShone
a144b81c4a
Update Autocompletion Generation Documentation
...
The autocompletion command has been changed to generate. Update documentation to reflect this change.
2018-04-12 11:50:26 -04:00
Alexander Neumann
3c453a4217
Merge pull request #1715 from rawtaz/patch-1
...
Add "Including and Exluding Files" heading in docs
2018-04-11 21:27:22 +02:00
rawtaz
1e2f23d77a
Add "Including and Exluding Files" heading in docs
...
Adds "Including and Exluding Files" heading in the backup section in the docs.
[ci skip]
2018-04-11 21:14:26 +02:00
Alexander Neumann
2c76e724ab
Merge pull request #1712 from restic/list-ignore-non-existing-dirs
...
Improve handling non-existing dirs in List()
2018-04-10 21:54:00 +02:00
Alexander Neumann
577faa7570
local/sftp: Handling non-existing dirs in List()
2018-04-10 21:35:30 +02:00
Alexander Neumann
6a34e0d10f
Merge pull request #1709 from restic/improve-check-errors
...
Improve error message for orphaned pack files
2018-04-07 13:38:41 +02:00
Alexander Neumann
b08f21cdc6
Add entry to changelog
2018-04-07 13:05:44 +02:00
Alexander Neumann
1c1fede399
Improve error message for orphaned pack files
2018-04-07 10:07:54 +02:00
Alexander Neumann
63a0913e6e
Merge pull request #1705 from alirazeen/doc-add-scripting
...
Add scripting section to docs
2018-04-07 09:47:17 +02:00
Ali Razeen
325957443e
Add scripting section to docs
2018-04-04 10:43:20 -04:00
Alexander Neumann
4b2d3b15a2
Add entry to changelog
2018-04-03 20:42:06 +02:00
Alexander Neumann
4e2a87c920
Merge pull request #1703 from ebastos/issue1608
...
Fixed issue #1608 - Use --time argument properly
2018-04-03 20:40:41 +02:00
Eri Bastos
901e1b129c
Fixed issue #1608 - Use --time argument properly
...
Backups via stdin will now handle --time argument and pass it down as
expected
2018-04-03 14:40:42 -03:00
Alexander Neumann
4478d633e2
Merge pull request #1702 from restic/update-simple-scrypt
...
Update github.com/elithrar/simple-scrypt
2018-04-02 20:07:18 +02:00
Alexander Neumann
92f516b1d4
Update github.com/elithrar/simple-scrypt
...
For details see #1697
2018-04-02 19:48:25 +02:00
Alexander Neumann
03193e6d92
Fix changelog entries
2018-04-02 12:42:48 +02:00
Alexander Neumann
01fe719aff
check: Make sure temp cache dir is removed
2018-04-01 18:09:53 +02:00
Alexander Neumann
2c964df3e2
Merge pull request #1699 from restic/fix-incremental-backup-test
...
Slightly increment size for TestIncrementalBackup
2018-04-01 14:33:12 +02:00
Alexander Neumann
8919125b0b
Merge pull request #1696 from restic/fix-check-cache
...
check: Improve cache handling
2018-04-01 14:33:00 +02:00
Alexander Neumann
1f5137aa70
Add entry to CHANGELOG
2018-04-01 13:59:27 +02:00
Alexander Neumann
a95eb33616
check: Use cache in temporary directory if possible
...
Closes #1694
2018-04-01 13:59:27 +02:00
Alexander Neumann
e68a7fea8a
check: Allow filling the cache during check
...
Closes #1665
2018-04-01 13:59:27 +02:00
Alexander Neumann
2e7ec717c1
repository: Move cache preparation into function
2018-04-01 13:59:27 +02:00
Alexander Neumann
22d5061df2
Merge pull request #1698 from restic/reduce-backend-tests-travis
...
Travis: Skip cloud backend tests for most Go versions
2018-04-01 13:58:38 +02:00
Alexander Neumann
4544a77172
Slightly increment size for TestIncrementalBackup
...
This should make the test more reliable, it should hit the accidental
"repo is has grown too much" way less often.
2018-04-01 13:49:42 +02:00
Alexander Neumann
b3a073e066
Travis: Skip cloud backend tests for most Go versions
2018-04-01 13:13:38 +02:00
Alexander Neumann
b077a1227b
Merge pull request #1657 from restic/rclone-backend
...
Rclone backend
2018-04-01 10:56:10 +02:00
Alexander Neumann
3f48e0e0f4
Add extra options to rclone
...
For details see https://github.com/restic/restic/pull/1657#issuecomment-377707486
2018-04-01 10:34:30 +02:00
Alexander Neumann
86f4b03730
Remove unneeded byte counters
2018-04-01 10:18:38 +02:00
Alexander Neumann
c43c94776b
rclone: Make concurrent connections configurable
2018-04-01 10:18:38 +02:00
Alexander Neumann
0b776e63e7
backend/rclone: Request random file name
...
When `/` is requested, rclone returns the list of all files in the
remote, which is not what we want (and it can take quite some time).
2018-04-01 10:18:38 +02:00
Alexander Neumann
360ff1806a
doc: Fix instructions for rclone backend
2018-04-01 10:18:38 +02:00
Alexander Neumann
1beeb7d0dd
doc/REST: Make documentation match reality
2018-04-01 10:18:38 +02:00
Alexander Neumann
e978b36713
doc: Add hint how to debug rclone
2018-04-01 10:18:38 +02:00
Alexander Neumann
737d93860a
Extend first timeout to 60 seconds.
2018-04-01 10:18:38 +02:00
Alexander Neumann
011217e4bf
backend/rclone: Improve documentation and README
2018-04-01 10:18:38 +02:00
Alexander Neumann
362d5afec4
Add entry to changelog
2018-04-01 10:18:38 +02:00
Alexander Neumann
4172fcd167
doc: Add rclone backend
2018-04-01 10:18:38 +02:00
Alexander Neumann
518bf4e5f6
doc: Correct verbatim text in the manual
2018-04-01 10:18:38 +02:00
Alexander Neumann
17312d3a98
backend/rest: Ensure base URL ends with slash
...
This makes it easier for rclone.
2018-04-01 10:18:38 +02:00
Alexander Neumann
4d5c7a8749
backend/rclone: Make sure rclone terminates
2018-04-01 10:18:38 +02:00
Alexander Neumann
fc0295016a
Address code review comments
2018-04-01 10:18:38 +02:00
Alexander Neumann
99b62c11b8
backend/rclone: Stop rclone in case of errors
2018-04-01 10:18:38 +02:00
Alexander Neumann
6d9a029e09
backend/rclone: Prefix all error messages
2018-04-01 10:18:38 +02:00
Alexander Neumann
20352886f3
Update Gopkg.lock
2018-04-01 10:18:38 +02:00
Alexander Neumann
3622b60c13
CI: Check that rclone backend test isn't skipped
2018-04-01 10:16:31 +02:00
Alexander Neumann
065fe1e54f
backend/rclone: Skip test if binary is unavailable
2018-04-01 10:16:31 +02:00
Alexander Neumann
4dc0f24b38
backend/tests: Drain reader before returning error
2018-04-01 10:16:31 +02:00
Alexander Neumann
fe99340e40
Add rclone backend
2018-04-01 10:16:31 +02:00
Alexander Neumann
e377759c81
rest: Export Backend struct
2018-04-01 10:16:31 +02:00
Alexander Neumann
61f6db25f4
CI: install rclone
2018-04-01 10:16:31 +02:00
Alexander Neumann
cabbbd2b14
backend/rest: Export Content-Types
2018-04-01 10:16:31 +02:00
Alexander Neumann
cf4cf94418
Move backend/sftp.StartForeground to backend/
2018-04-01 10:16:31 +02:00
Alexander Neumann
34f27edc03
Refactor SplitShellStrings
2018-04-01 10:16:31 +02:00
Alexander Neumann
345b6c4694
Move backend/sftp.SplitShellArgs to backend/
2018-04-01 10:16:31 +02:00
Alexander Neumann
e4a39e02d2
Merge pull request #1695 from xulongwu4/patch-2
...
Add instructions for installation on Solus
2018-03-31 09:14:25 +02:00
xulongwu4
432e167255
Add instructions for installation on Solus
2018-03-30 17:36:29 -04:00
Alexander Neumann
594256bfa4
Merge pull request #1693 from restic/update-deps
...
Update dependencies
2018-03-30 17:19:08 +02:00
Alexander Neumann
0fcb1e6b7a
Merge pull request #1692 from restic/print-forget-policy
...
forget: Print policy
2018-03-30 17:19:05 +02:00
Alexander Neumann
38795c66c9
Update vendored library gopkg.in/yaml.v2
2018-03-30 12:53:13 +02:00
Alexander Neumann
c0960f538f
Update vendored library google.golang.org/api
2018-03-30 12:51:18 +02:00
Alexander Neumann
5b6568875c
Update vendored library golang.org/x/text
2018-03-30 12:50:04 +02:00
Alexander Neumann
d8dd79eb0b
Update vendored library golang.org/x/sys
2018-03-30 12:48:49 +02:00
Alexander Neumann
2bdeb645b9
Update vendored library golang.org/x/sync
2018-03-30 12:47:30 +02:00
Alexander Neumann
9f2ffa3e50
Update vendored library golang.org/x/oauth2
2018-03-30 12:46:26 +02:00
Alexander Neumann
d4bab5c133
Update vendored library golang.org/x/net
2018-03-30 12:45:07 +02:00
Alexander Neumann
3473d73d0c
Update vendored library github.com/spf13/cobra
2018-03-30 12:43:03 +02:00
Alexander Neumann
917cc542c9
Update vendored library github.com/sirupsen/logrus
2018-03-30 12:41:46 +02:00
Alexander Neumann
a9cf5d482a
Update vendored library github.com/russross/blackfriday
2018-03-30 12:40:05 +02:00
Alexander Neumann
75946e7c58
Update vendored library github.com/pkg/xattr
2018-03-30 12:38:37 +02:00
Alexander Neumann
19035e977b
Update vendored library github.com/pkg/sftp
2018-03-30 12:37:16 +02:00
Alexander Neumann
d9ba9279e0
Update vendored library github.com/ncw/swift
2018-03-30 12:35:13 +02:00
Alexander Neumann
31e156c666
Update vendored library github.com/minio/minio-go
2018-03-30 12:33:40 +02:00
Alexander Neumann
7e6fff324c
Update vendored library golang.org/x/crypto
2018-03-30 12:26:26 +02:00
Alexander Neumann
e94d2da890
Update vendored library github.com/golang/protobuf
2018-03-30 11:52:18 +02:00
Alexander Neumann
874b3dbbd9
Update vendored library github.com/go-ini/ini
2018-03-30 11:51:01 +02:00
Alexander Neumann
0d01c27c9e
Update vendored library github.com/dgrijalva/jwt-go
2018-03-30 11:49:18 +02:00
Alexander Neumann
30110fcfc2
Update vendored library github.com/cpuguy83/go-md2man
2018-03-30 11:48:16 +02:00
Alexander Neumann
673f0bbd6c
Update vendored library github.com/cenkalti/backoff
2018-03-30 11:45:27 +02:00
Alexander Neumann
5a77b2ab49
Update vendored library github.com/Azure/azure-sdk-for-go
2018-03-30 11:42:11 +02:00
Alexander Neumann
a951e7b126
Update vendored library cloud.google.com/go
2018-03-30 11:41:12 +02:00
Alexander Neumann
d3f9c8b362
forget: Print policy
2018-03-30 10:24:26 +02:00
Alexander Neumann
a4ff591165
Update URL to template
2018-03-25 11:55:37 +02:00
Alexander Neumann
49dd70c771
Merge pull request #1686 from gtrafimenkov/minor-spelling-fixes
...
Correct spelling mistakes
2018-03-25 11:54:53 +02:00
Gennady Trafimenkov
a64f24029b
Correct spelling mistakes
2018-03-25 00:42:33 +03:00
Alexander Neumann
0886738d24
Add entry to CHANGELOG
2018-03-24 18:40:49 +01:00
Alexander Neumann
9fc38803e0
Merge pull request #1684 from restic/fix-rest-tests
...
Fix rest-server tests
2018-03-24 18:39:44 +01:00
Alexander Neumann
e5c929b793
Fix rest-server tests
...
Since today, the rest-server needs to be explicitly told (via
`--no-auth`) that authentication is not necessary.
2018-03-24 18:06:21 +01:00
Alexander Neumann
0e0fee9c8f
Update changelog template
2018-03-24 17:31:21 +01:00
Alexander Neumann
26769a39eb
Merge pull request #1679 from rawtaz/version-oneliner
...
Make version output one line.
2018-03-23 21:14:37 +01:00
Leo R. Lundgren
923be90906
Make version output one line.
2018-03-21 21:49:03 +01:00
Alexander Neumann
84a22eac92
Merge pull request #1675 from oliver/doc-fixes
...
Minor doc fixes/improvements
2018-03-21 20:54:51 +01:00
Alexander Neumann
6eb1be0be4
Fix changelog files
2018-03-21 20:53:11 +01:00
Alexander Neumann
f31bbcf1a9
CI: Return error when calens fails
2018-03-21 20:53:01 +01:00
Alexander Neumann
5d09fca6a2
Merge pull request #1676 from bowensong/quiet-skip-scan
...
Skip archiver.Scan before backup when --quiet is set
2018-03-21 20:45:34 +01:00
Bowen Song
34671d7c9b
Skip archiver.Scan before backup when --quiet is set
2018-03-20 20:44:10 +00:00
Alexander Neumann
4a524da736
FAQ: Add suggestion on how to solve sftp error
2018-03-18 21:39:11 +01:00
Oliver Gerlich
e361cc3807
040_backup.rst: add note regarding paths in --files-from argument
2018-03-18 20:51:04 +01:00
Alexander Neumann
3cd8a7bc96
Fix small things left open by #1552
2018-03-18 19:54:12 +01:00
Alexander Neumann
8206f85d2e
Merge pull request #1552 from lawrencejones/use-auto-auth
...
Automatically load Google auth
2018-03-18 19:53:30 +01:00
Alexander Neumann
7022144e0f
Merge pull request #1673 from rawtaz/doc-precompiled
...
doc: Add more info about using pre-compiled builds.
2018-03-18 19:50:25 +01:00
Oliver Gerlich
1bee3e01fa
040_backup.rst: fix typo
2018-03-18 17:06:21 +01:00
Oliver Gerlich
624a2d8305
040_backup.rst: quote wildcard characters
...
This doesn't really matter for --exclude in most cases, but it avoids
spreading bad ideas.
2018-03-18 17:05:53 +01:00
Lawrence Jones
57c6233982
dep ensure
2018-03-16 10:31:30 +00:00
Leo R. Lundgren
c161aba084
doc: Add more info about using pre-compiled builds.
2018-03-15 21:23:07 +01:00
Alexander Neumann
0279fd7212
Merge pull request #1669 from restic/make-tests-faster
...
Reduce test runtime
2018-03-13 19:50:34 +01:00
Alexander Neumann
dedf17f5e8
Merge pull request #1650 from copart/patch-1
...
Added copr repositories for RHEL/Centos/Fedora
2018-03-11 21:52:18 +01:00
Alexander Neumann
817890794d
Merge pull request #1668 from restic/fix-1663
...
Return the first password/key which works
2018-03-11 21:51:54 +01:00
Alexander Neumann
b9ada91054
Reduce data for TestCreateSnapshot
2018-03-11 21:42:39 +01:00
Alexander Neumann
dfb6d0fced
Reduce data for TestIncrementalBackup
2018-03-11 21:17:27 +01:00
Alexander Neumann
c6c1dccc53
Reduce data set for TestRestore
2018-03-11 21:10:37 +01:00
Alexander Neumann
279566bafe
Reduce dataset for integration tests
2018-03-11 21:07:47 +01:00
Alexander Neumann
c67a8452f7
Disable polynomial check for chunker for tests
2018-03-11 20:59:40 +01:00
Alexander Neumann
5253ef218c
Remove TestParallelSaveWithDuplication
2018-03-11 19:49:48 +01:00
Alexander Neumann
0923976909
Remove TestArchiverDuplication
2018-03-11 19:44:25 +01:00
Lawrence Jones
492baf991f
Update docs and add changelog entry: Google auth
...
Add documentation around using default Google application credentials,
along with a changelog extra that describes the feature and the
potential impact on existing restic uses (read: none).
2018-03-11 17:12:30 +00:00
Lawrence Jones
0dfdc11ed9
Automatically load Google auth
...
This change removes the hardcoded Google auth mechanism for the GCS
backend, instead using Google's provided client library to discover and
generate credential material.
Google recommend that client libraries use their common auth mechanism
in order to authorise requests against Google services. Doing so means
you automatically support various types of authentication, from the
standard GOOGLE_APPLICATION_CREDENTIALS environment variable to making
use of Google's metadata API if running within Google Container Engine.
2018-03-11 17:11:25 +00:00
Alexander Neumann
54c6837ec4
Merge pull request #1651 from qbit/obsd_inst
...
doc: Add install instructions for OpenBSD
2018-03-11 14:49:40 +01:00
Alexander Neumann
e085713b35
Return the first password/key which works
...
Closes #1663
2018-03-11 14:12:21 +01:00
Alexander Neumann
e77d8c64a7
Merge pull request #1661 from restic/fix-rest-content-length
...
rest: Really set Content-Length HTTP header
2018-03-10 20:34:30 +01:00
Alexander Neumann
a410fa16a1
Merge pull request #1667 from restic/improve-error-config
...
Return error message for config decryption failure
2018-03-10 20:34:25 +01:00
Alexander Neumann
b3e1089cf9
Return error message for config decryption failure
...
See #1663
2018-03-09 21:05:35 +01:00
Alexander Neumann
7f8e269891
Merge pull request #1662 from ebastos/version_password
...
Skip checking for password file issue #1632
2018-03-09 20:42:32 +01:00
Alexander Neumann
fcc9ce81ba
rest: Really set Content-Length HTTP header
2018-03-09 20:21:34 +01:00
Eri Bastos
b9d643358a
Skip checking for password file existence if command is 'version' - Issue #1632
2018-03-08 17:55:03 -04:00
Alexander Neumann
ab5ef600a2
Merge pull request #1660 from ncw/rest-fix
...
backend/rest: check HTTP error response for List
2018-03-08 21:55:58 +01:00
Nick Craig-Wood
04c4033695
backend/rest: check HTTP error response for List
...
Before this change restic would attempt to JSON decode the error
message resulting in confusing `Decode: invalid character 'B' looking
for beginning of value` messages. Afterwards it will return `List
failed, server response: 400 Bad Request (400)`
2018-03-08 10:22:43 +00:00
Alexander Neumann
de37b68baa
Move all unreleased changelog entries
2018-03-05 21:31:52 +01:00
Alexander Neumann
bdc206d440
Remove unneeded mkdir
2018-03-05 21:17:57 +01:00
Alexander Neumann
efe2e792b3
Correct changelog entries
2018-03-05 21:17:52 +01:00
Alexander Neumann
6f3c23eba7
Merge pull request #1653 from restic/fix-1652
...
lock: Ignore invalid lock file
2018-03-05 20:49:03 +01:00
Alexander Neumann
4b34bc3210
Reformat changelog entry
2018-03-05 20:40:40 +01:00
Alexander Neumann
6ed9100aa1
Add version number hint for the changelog entry
2018-03-05 20:34:07 +01:00
Alexander Neumann
c63b02d0f1
Move changelog entries to unreleased version
2018-03-05 20:32:10 +01:00
Alexander Neumann
d0205ec889
Add entry to changelog
2018-03-05 20:22:45 +01:00
Alexander Neumann
d8dcbc89d1
lock: Ignore invalid lock file
...
This commit fixes a bug introduced in
e9ea26884739a218bbab0248b634d2821be9c3ea: When an invalid lock is
encountered (e.g. if the file is empty), the code used to ignore that,
but now returns the error.
Now, invalid files are ignored for the normal lock check, and removed
when `restic unlock --remove-all` is run.
Closes #1652
2018-03-05 20:19:57 +01:00
Alexander Neumann
be0a5b7f06
Merge pull request #1649 from jasperla/solaris
...
Minimal set of patches to get restic working on Solaris
2018-03-05 20:00:17 +01:00
Aaron Bieber
24ce08e122
doc: Add install instructions for OpenBSD
2018-03-05 06:31:57 -07:00
copart
864eaeab7c
Added copr repositories for RHEL/Centos/Fedora
2018-03-04 15:50:30 -05:00
Jasper Lievisse Adriaanse
96311d1a2b
Add support for illumos/Solaris
...
This does come without xattr/fuse support at this point.
NB: not hooking up the integration tests as restic won't compile without
cgo with Go < 1.10.
2018-03-04 20:11:29 +00:00
Alexander Neumann
da77f4a2e2
Merge pull request #1647 from duzvik/aws-session-token
...
Change priority of AWS credential providers to accept AWS_SESSION_TOKEN
2018-03-04 20:54:56 +01:00
denis.uzvik
6bb1bcce03
Change priority of AWS credential providers to accept AWS_SESSION_TOKEN
2018-03-04 19:58:27 +02:00
Alexander Neumann
6edf28d1e1
Merge pull request #1639 from restic/improve-backend-save
...
backend: Improve/Cleanup Save()
2018-03-04 13:35:50 +01:00
Alexander Neumann
929afc63d5
Use int64 for the length in the RewindReader
2018-03-04 10:40:42 +01:00
Alexander Neumann
99f7fd74e3
backend: Improve Save()
...
As mentioned in issue [#1560 ](https://github.com/restic/restic/pull/1560#issuecomment-364689346 )
this changes the signature for `backend.Save()`. It now takes a
parameter of interface type `RewindReader`, so that the backend
implementations or our `RetryBackend` middleware can reset the reader to
the beginning and then retry an upload operation.
The `RewindReader` interface also provides a `Length()` method, which is
used in the backend to get the size of the data to be saved. This
removes several ugly hacks we had to do to pull the size back out of the
`io.Reader` passed to `Save()` before. In the `s3` and `rest` backend
this is actively used.
2018-03-03 15:49:44 +01:00
Alexander Neumann
58306bfabb
Merge pull request #1648 from duzvik/s3-bucketexists
...
Ignore s3 AccessDenied error, during creation of repository
2018-03-02 22:38:33 +01:00
denis.uzvik
f6890210aa
Add entry to changelog
2018-03-02 11:06:06 +02:00
denis.uzvik
5873ab4031
Ignore s3 AccessDenied error, during creation of repository
2018-03-02 10:47:20 +02:00
Alexander Neumann
ab7a3a803d
Update build.go from github.com/fd0/build-go
2018-02-28 21:19:28 +01:00
Alexander Neumann
1e868933c5
build.go: Allow specifying the temp dir to use
2018-02-27 21:56:42 +01:00
Alexander Neumann
21f67a0a13
Fix linebreak in GitHub changelog template
2018-02-26 21:41:37 +01:00
Alexander Neumann
272ccec7e1
Add VERSION for 0.8.3
2018-02-26 21:32:16 +01:00
Alexander Neumann
68bf1509bd
Update manpages and auto-completion
2018-02-26 21:32:16 +01:00
Alexander Neumann
cfccd67600
Generate CHANGELOG.md for 0.8.3
2018-02-26 21:32:07 +01:00
Alexander Neumann
bc461d32e0
Add release date for 0.8.3
2018-02-26 21:31:50 +01:00
Alexander Neumann
ee4bfdf954
changelog: Fix spelling
2018-02-26 21:27:32 +01:00
Alexander Neumann
3037894f62
Add entry to changelog
2018-02-26 21:20:41 +01:00
Alexander Neumann
89075bdf6d
Merge pull request #1643 from restic/fix-1641
...
Ignore files in the repo with invalid names
2018-02-26 21:18:05 +01:00
Alexander Neumann
c323f73bf9
Ignore files in the repo with invalid names
...
Closes #1641
2018-02-26 20:53:38 +01:00
Alexander Neumann
aef5e03731
Merge pull request #1638 from restic/fix-list-retry
...
backend/retry: return worker function error and abort
2018-02-25 21:20:08 +01:00
Alexander Neumann
fc1f74d32d
Merge pull request #1640 from restic/fix-1637
...
mount: Ignore non-existing locks
2018-02-25 14:00:04 +01:00
Alexander Neumann
7d59df1ab8
mount: Ignore non-existing locks
...
Closes #1637
2018-02-25 13:11:03 +01:00
Alexander Neumann
2866f3f31c
Add pull request to changelog entry
2018-02-24 14:53:46 +01:00
Alexander Neumann
dc1154c8ad
Merge pull request #1556 from ifedorenko/check-subset
...
Add --read-data-subset flag to check command
2018-02-24 14:53:20 +01:00
Alexander Neumann
35a816e8ab
Add entry to changelog
2018-02-24 13:34:42 +01:00
Alexander Neumann
93210614f4
backend/retry: return worker function error and abort
...
This is a bug fix: Before, when the worker function fn in List() of the
RetryBackend returned an error, the operation is retried with the next
file. This is not consistent with the documentation, the intention was
that when fn returns an error, this is passed on to the caller and the
List() operation is aborted. Only errors happening on the underlying
backend are retried.
The error leads to restic ignoring exclusive locks that are present in
the repo, so it may happen that a new backup is written which references
data that is going to be removed by a concurrently running `prune`
operation.
The bug was reported by a user here:
https://forum.restic.net/t/restic-backup-returns-0-exit-code-when-already-locked/484
2018-02-24 13:26:13 +01:00
Alexander Neumann
dfd37afee2
Merge pull request #1636 from kurin/pack-header
...
Refactor the eager-header reads for readability.
2018-02-23 17:36:45 +01:00
Toby Burress
08a5281bd4
Incorporate PR review comments.
2018-02-22 17:37:10 +00:00
Toby Burress
cdb48a8970
Add tests for the eager-header refactor.
2018-02-22 01:14:04 +00:00
Toby Burress
4fd5f0b8a9
Refactor the eager-header reads for readability.
...
This pulls the header reads into a function that works in terms of the
number of records requested. This preserves the existing logic of
initially reading 15 records and then falling back if that fails.
In the event of a header with more than 15 records, it will read all
records, including the already-seen final 15 records.
2018-02-22 00:45:40 +00:00
Alexander Neumann
92ad6bf74f
Add pull request to changelog
2018-02-21 19:52:16 +01:00
Alexander Neumann
2c7dd3edf4
Merge pull request #1635 from ifedorenko/1633-negative-load-offset
...
Fixed unexpected 'pack file cannot be listed' error
2018-02-21 19:51:38 +01:00
Igor Fedorenko
19e7803ac6
Fixed unexpected 'pack file cannot be listed' error
...
Fixes #1633
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-20 21:28:57 -05:00
Alexander Neumann
9f0605766c
Add entry to changelog
2018-02-20 22:10:52 +01:00
Alexander Neumann
1a5d7a9965
Merge pull request #1634 from restic/update-blazer
...
Update github.com/kurin/blazer to 0.3.0
2018-02-20 22:01:30 +01:00
Alexander Neumann
296769355d
Update github.com/kurin/blazer to 0.3.0
...
This commit will reduce the number of HTTP requests per file uploaded
from two to one.
2018-02-20 21:01:21 +01:00
Igor Fedorenko
07d080830e
Add --read-data-subset flag to check command
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-18 23:31:27 -05:00
Alexander Neumann
c99eabfb37
Merge pull request #1625 from restic/update-blazer
...
Update B2 client library (github.com/kurin/blazer)
2018-02-18 19:18:52 +01:00
Alexander Neumann
842fe43590
Update github.com/kurin/blazer to 0.2.2
2018-02-18 14:53:23 +01:00
Alexander Neumann
be02008025
Merge pull request #1611 from qbit/master
...
On OpenBSD only root can set sticky bit
2018-02-18 12:57:54 +01:00
Alexander Neumann
29da86b473
Merge pull request #1623 from restic/backend-relax-restrictions
...
backend: Relax requirement for new files
2018-02-18 12:56:52 +01:00
Alexander Neumann
bad7215696
Add entry to CHANGELOG
2018-02-18 12:04:44 +01:00
Alexander Neumann
881ff5e554
Move changelog file, improve text
2018-02-18 11:51:45 +01:00
Alexander Neumann
86b7fd0335
Merge pull request #1624 from rawtaz/patch-1
...
Attempt to make issue template a bit clearer
2018-02-18 11:50:11 +01:00
rawtaz
70209d7d1d
End both list items with a dot.
...
Changed my mind after checking other lists in the project's files, ending with a dot seems to be preferred here, and I like that better too.
2018-02-17 23:47:40 +01:00
rawtaz
f07552161c
Remove potentially excessive dot
...
Some people like list items to not end with a comma or dot, some like it when they do. To keep things like and coherent I removed them in this case.
2018-02-17 23:40:30 +01:00
rawtaz
856f3a9135
Add forum URL back
2018-02-17 23:36:47 +01:00
Alexander Neumann
49e9bcadb7
Merge pull request #1560 from ifedorenko/1559-load-error-handling
...
Retry all repository file download errors
2018-02-17 23:25:28 +01:00
rawtaz
1b8823ef2e
Attempt to make issue template a bit clearer
2018-02-17 23:19:58 +01:00
Alexander Neumann
b5062959c8
backend: Relax requirement for new files
...
Before, all backend implementations were required to return an error if
the file that is to be written already exists in the backend. For most
backends, that means making a request (e.g. via HTTP) and returning an
error when the file already exists.
This is not accurate, the file could have been created between the HTTP
request testing for it, and when writing starts. In addition, apart from
the `config` file in the repo, all other file names have pseudo-random
names with a very very low probability of a collision. And even if a
file name is written again, the way the restic repo is structured this
just means that the same content is placed there again. Which is not a
problem, just not very efficient.
So, this commit relaxes the requirement to return an error when the file
in the backend already exists, which allows reducing the number of API
requests and thereby the latency for remote backends.
2018-02-17 22:39:18 +01:00
Alexander Neumann
2fb4d44a4d
Add VERSION for 0.8.2
2018-02-17 20:27:28 +01:00
Alexander Neumann
3c08f956a3
Update manpages and auto-completion
2018-02-17 20:27:28 +01:00
Alexander Neumann
1463a7aaaa
Generate CHANGELOG.md for 0.8.2
2018-02-17 20:27:17 +01:00
Alexander Neumann
76b8e080c3
changelog: Add release date for 0.8.2
2018-02-17 20:26:43 +01:00
Alexander Neumann
a9f9243afc
scripts/release: Fix usage for restic/builder
...
The restic/builder Docker container now has a slightly different usage.
2018-02-17 19:10:32 +01:00
Alexander Neumann
34bf9de339
Fix Go versions for Travis
2018-02-17 18:48:15 +01:00
Alexander Neumann
597f6be4e4
Disable CI tests on Go 1.8 on OS X
2018-02-17 18:39:16 +01:00
Alexander Neumann
aeeec0ff94
Update Go version for CI tests
2018-02-17 18:37:37 +01:00
Igor Fedorenko
ab040d8811
Introduced repository.DownloadAndHash helper
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-16 21:13:11 -05:00
Igor Fedorenko
d58ae43317
Reworked Backend.Load API to retry errors during ongoing download
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-16 21:12:14 -05:00
Alexander Neumann
b723094739
scripts/release: Check changelog subdir with date
2018-02-16 23:13:36 +01:00
Alexander Neumann
84dcc3c396
Update changelog/ for new version of calens
...
We've reworked calens[1] a bit, the changelog/releases file is not
needed any more. Insteady, the release date is stored in the dir name
which contains the changelog entries.
[1] https://github.com/restic/calens
2018-02-16 23:05:58 +01:00
Alexander Neumann
21747bc219
Remove punctuation from changelog entry title
2018-02-16 22:49:24 +01:00
Alexander Neumann
28b00891d2
scripts/release: Check Go version in container
2018-02-16 22:35:53 +01:00
Alexander Neumann
b4e00c324f
scripts/release: Check release date first
2018-02-16 21:52:34 +01:00
Alexander Neumann
4eb9df63cf
Merge pull request #1616 from rawtaz/fix-backup-doc
...
Restructure and complement exclude options doc
2018-02-14 22:11:14 +01:00
Alexander Neumann
fde674d214
Merge pull request #1615 from rawtaz/fix-forget-doc
...
Fix #1519 and formatting in `forget` doc
2018-02-14 22:11:10 +01:00
rawtaz
fbb08c14de
Note that multiple `--exclude-file` may be used.
2018-02-14 22:05:52 +01:00
Leo R. Lundgren
8b95b48bae
Fix grammar and formatting.
2018-02-14 02:59:47 +01:00
Leo R. Lundgren
71a4ccea71
Document refusal to process empty forget policy.
2018-02-14 02:59:41 +01:00
Leo R. Lundgren
5a9c7c51e6
Fix --keep-* options list formatting.
2018-02-14 02:59:33 +01:00
rawtaz
5abd133e26
Restructure and complement exclude options doc
2018-02-14 02:03:04 +01:00
Alexander Neumann
2debb5c0e0
Merge pull request #1613 from restic/strip-space-files-from
...
Strip spaces from items read via --files-from
2018-02-11 21:35:23 +01:00
Alexander Neumann
54c0794cf3
Merge pull request #1603 from jfparis/master
...
Add support for GOARM parameter when cross compiling
2018-02-11 21:03:29 +01:00
Alexander Neumann
cff3f3dc0d
Add entry to CHANGELOG
2018-02-11 21:00:38 +01:00
Alexander Neumann
c5575c7ed9
Strip spaces from items read via --files-from
...
In #1590 , it was mentioned that while lines read from exclude files via
`--exclude-file` have leading and trailing spaces stripped, this is not
the case for lines read via `--files-from`. This commit fixes that,
spaces are always stripped.
2018-02-11 20:56:11 +01:00
Aaron Bieber
99d88ad297
Disable the 'testSticky' test on OpenBSD. Only root can set sticky.
2018-02-11 07:46:31 -07:00
Alexander Neumann
7f0d964a05
Update build.go
2018-02-11 11:34:03 +01:00
Alexander Neumann
514f1b8917
Relax timeout backend test
2018-02-10 12:53:38 +01:00
Alexander Neumann
a7a99be460
Correct forum host name in issue template
...
Closes #1609
2018-02-10 11:13:22 +01:00
Alexander Neumann
e17f65b23a
Update logo
2018-02-05 20:48:48 +01:00
Jean-François Paris
e86fb5eedd
Add support for GOARM parameter when cross compiling
...
Go can cross compile to older ARM architectures (e.g. v5). In order to
do so one needs to use the GOARM parameter as documented in
https://github.com/golang/go/wiki/GoArm
2018-02-04 14:15:12 +00:00
Alexander Neumann
c8cb9a9509
Merge pull request #1579 from ifedorenko/RetryBackend-List
...
Implement RetryBackend.List()
2018-01-30 21:39:21 +01:00
Igor Fedorenko
aa333f4d49
Implement RetryBackend.List()
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-29 22:14:12 -05:00
Alexander Neumann
c40002246d
Merge pull request #1598 from Phaeilo/s3-tutorial
...
Update AWS S3 tutorial, fixing #1593
2018-01-29 21:52:32 +01:00
Philip Huppert
77a6054d64
Update AWS S3 tutorial, fixing #1593
2018-01-29 20:47:27 +01:00
Alexander Neumann
94ec55ff2c
Merge pull request #1595 from restic/remove-read-bandwidth
...
Remove archiver progress "data processed" bandwith
2018-01-28 10:15:53 +01:00
Alexander Neumann
2369da158f
Merge pull request #1592 from ncw/helpful-tests
...
Make backend tests more helpful
2018-01-28 10:09:35 +01:00
Alexander Neumann
5379950c64
Add entry to changelog
2018-01-28 09:36:55 +01:00
Alexander Neumann
cbfa516575
Remove archiver progress "data processed" bandwith
...
This commit removes the bandwidth displayed during backup process. It is
misleading and seldomly correct, because it's neither the "read
bandwidth" (only for the very first backup) nor the "upload bandwidth".
Many users are confused about (and rightly so), c.f. #1581 , #1033 , #1591
We'll eventually replace this display with something more relevant when
#1494 is done.
2018-01-28 09:30:26 +01:00
Nick Craig-Wood
fb62da1748
Make backend tests more helpful
...
* In TestList check that backend is empty first
* Improve error message in TestBackend
2018-01-27 21:36:35 +00:00
Alexander Neumann
b1b1f6e04f
Merge pull request #1594 from restic/gcs-use-http-transport
...
GS: Use generic http transport
2018-01-27 22:16:49 +01:00
Alexander Neumann
05f826a4fc
Update Gopkg.lock
2018-01-27 21:48:41 +01:00
Alexander Neumann
649afb1198
Add entry to changelog
2018-01-27 20:16:51 +01:00
Alexander Neumann
5dc8d3588d
GS: Use generic http transport
...
During the development of #1524 I discovered that the Google Cloud
Storage backend did not yet use the HTTP transport, so things such as
bandwidth limiting did not work. This commit does the necessary magic to
make the GS library use our HTTP transport.
2018-01-27 20:12:34 +01:00
Alexander Neumann
9d2aa0a71c
Merge pull request #1524 from brycied00d/https_client_certificate
...
Support for TLS client certificate authentication
2018-01-27 20:12:09 +01:00
Alexander Neumann
c34db983d8
Read TLS client cert and key from the same file
2018-01-27 14:02:01 +01:00
Bryce Chidester
e805b968b1
Support for TLS client certificate authentication
...
This adds --tls-client-cert and --tls-client-key parameters and enables use
of that certificate/key pair when connecting to https servers.
2018-01-27 13:18:22 +01:00
Alexander Neumann
e706f1a8d1
Merge pull request #1589 from restic/fix-intermediate-index-upload
...
archiver: Fix intermediate index upload
2018-01-27 12:22:11 +01:00
Alexander Neumann
4219bfbcc9
Add entry to changelog
2018-01-26 22:05:53 +01:00
Alexander Neumann
f99c95c766
archiver: Fix intermediate index upload
...
A user discovered[1] that when the backup finishes during the upload of
an intermediate index, the upload is cancelled and the index never fully
saved, but the snapshot is saved and the backup finalizes without an
error. This lead to a situation where a snapshot references data that is
contained in the repo, but not referenced in any index, leading to
strange error messages.
This commit uses a dedicated context to signal the intermediate index
uploading routine to terminate after the last index has been uploaded.
This way, an upload running when the backup finishes is completed before
the routine terminates and the snapshot is saved.
[1] https://forum.restic.net/t/error-loading-tree-check-prune-and-forget-gives-error-b2-backend/406
2018-01-26 22:01:07 +01:00
Alexander Neumann
cccb2fc7e7
Merge pull request #1583 from restic/close-open-backend-files
...
Close backend files in case of errors
2018-01-26 21:57:28 +01:00
Alexander Neumann
ff3de66ddf
Merge pull request #1582 from restic/optimize-debug-log
...
Optimize debug logs
2018-01-26 21:57:18 +01:00
Alexander Neumann
fb0d01ef1f
Add pull request URL to changelog
2018-01-26 21:56:59 +01:00
Alexander Neumann
e11a183578
Merge pull request #1588 from restic/fix-sftp-without-tty
...
sftp: Allow running ssh without a tty
2018-01-26 21:56:41 +01:00
Alexander Neumann
ee6732d2c6
Add entry to changelog
2018-01-26 21:56:29 +01:00
Alexander Neumann
4b2f2b542d
Merge pull request #1584 from restic/limit-index-file-size
...
Limit index file size
2018-01-26 21:53:12 +01:00
Alexander Neumann
6eb2d76435
index: Lower parallel load to 4
2018-01-26 21:10:38 +01:00
Alexander Neumann
f63c7048c7
index: Limit index files to 3000 pack files
2018-01-26 21:07:16 +01:00
Alexander Neumann
ca3c566a99
index: Check structure after saving a new index
2018-01-26 21:07:16 +01:00
Alexander Neumann
7719cf88d9
b2: Check timeout
2018-01-26 21:07:05 +01:00
Alexander Neumann
00e905ebe6
sftp: Allow running ssh without a tty
2018-01-26 19:21:14 +01:00
Alexander Neumann
164acceb55
Appveyor: Update Go version and download URL
2018-01-25 21:09:23 +01:00
Alexander Neumann
909d9273cc
Close backend files in case of errors
2018-01-25 21:05:57 +01:00
Alexander Neumann
d62a664bb7
Merge pull request #1580 from ifedorenko/tests-errors-cause
...
Use errors.Cause in backend TestListCancel
2018-01-25 20:55:43 +01:00
Alexander Neumann
663c57ab4d
debug: Remove manual Str() call Log()
2018-01-25 20:49:41 +01:00
Alexander Neumann
ed99f53786
debug: Auto-shorten IDs passed as parameters
2018-01-25 20:49:26 +01:00
Igor Fedorenko
abc4027083
Use errors.Cause in backend TestListCancel
...
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-25 08:53:50 -05:00
Alexander Neumann
d6212ee2d9
Fix entry to changelog
2018-01-24 20:54:51 +01:00
Alexander Neumann
9c55e8d69c
Merge pull request #1549 from MJDSys/more_index_lookup_avoids
...
More optimizations to avoid calling Index.Lookup()
2018-01-24 20:53:30 +01:00
Alexander Neumann
1c9fca6d14
Combine changelog files for #1574 and #1575
2018-01-24 20:07:22 +01:00
Alexander Neumann
be205563be
Merge pull request #1575 from ifedorenko/1567_optimize-repository-ListPack
...
Optimize Repository.ListPack()
2018-01-24 20:01:15 +01:00
Alexander Neumann
fc6c341b26
Merge pull request #1574 from ifedorenko/1567_optimize-pack-readHeader
...
Optimize pack readHeader() implementation
2018-01-24 19:54:14 +01:00
Alexander Neumann
f86c141b83
Merge pull request #1577 from EdwardBetts/patch-1
...
Correct a spelling mistake.
2018-01-24 19:44:49 +01:00
Alexander Neumann
a6d4888d48
Merge pull request #1573 from restic/update-dps
...
Update dependencies
2018-01-24 19:43:32 +01:00
Alexander Neumann
44550a88a0
Merge pull request #1571 from restic/rework-backend-list
...
Rework backend list
2018-01-24 19:43:07 +01:00
Edward Betts
b5c23695c9
Correct a spelling mistake.
2018-01-24 12:09:41 +00:00
Igor Fedorenko
0084e42cb6
Optimize Repository.ListPack()
...
Use pack file size returned by Backend.List() to avoid extra per-pack
Backend.Stat() requests
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-23 22:39:51 -05:00
Matthew Dawson
fe33c05a20
debug/log: Add benchmarks for calling the logging function
...
Add some benchmarks for calling Log, both with a static string
along with calling the ID.Str and ID.String functions.
2018-01-23 22:30:00 -05:00
Matthew Dawson
3789e55e20
repostiory/index: Remove logging from Lookup function.
...
The logging in these functions double the time they take to execute.
However, it is only really useful on failures, which are better
reported by the calling functions.
benchmark old ns/op new ns/op delta
BenchmarkMasterIndexLookupSingleIndex-6 897 395 -55.96%
BenchmarkMasterIndexLookupMultipleIndex-6 2001 1090 -45.53%
BenchmarkMasterIndexLookupSingleIndexUnknown-6 492 215 -56.30%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6 1649 912 -44.69%
benchmark old allocs new allocs delta
BenchmarkMasterIndexLookupSingleIndex-6 9 1 -88.89%
BenchmarkMasterIndexLookupMultipleIndex-6 19 1 -94.74%
BenchmarkMasterIndexLookupSingleIndexUnknown-6 6 0 -100.00%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6 16 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkMasterIndexLookupSingleIndex-6 160 96 -40.00%
BenchmarkMasterIndexLookupMultipleIndex-6 240 96 -60.00%
BenchmarkMasterIndexLookupSingleIndexUnknown-6 48 0 -100.00%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6 128 0 -100.00%
2018-01-23 22:28:38 -05:00
Matthew Dawson
4cec7e236a
Add Changelog
2018-01-23 22:28:32 -05:00
Matthew Dawson
3a16148447
archiver/archiver: Use Index.Has() instead of Index.Lookup() in isKnownBlob
...
Index.Has() is a faster then Index.Lookup() for checking if a blob exists
in the index. As the returned data is never used, this avoids a ton
of allocations.
2018-01-23 22:26:10 -05:00
Matthew Dawson
df2c03a6a4
repository/master_index: Optimize Index.Lookup()
...
When looking up a blob in the master index, with several
indexes present in the master index, a significant amount of time
is spent generating errors for each failed lookup. However, these
errors are often used to check if a blob is present, but the contents
are not inspected making the overhead of the error not useful.
Instead, change Index.Lookup (and Index.LookupSize) to instead return
a boolean denoting if the blob was found instead of an error. Also change
all the calls to these functions to handle the new function signature.
benchmark old ns/op new ns/op delta
BenchmarkMasterIndexLookupSingleIndex-6 820 897 +9.39%
BenchmarkMasterIndexLookupMultipleIndex-6 12821 2001 -84.39%
BenchmarkMasterIndexLookupSingleIndexUnknown-6 5378 492 -90.85%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6 17026 1649 -90.31%
benchmark old allocs new allocs delta
BenchmarkMasterIndexLookupSingleIndex-6 9 9 +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6 59 19 -67.80%
BenchmarkMasterIndexLookupSingleIndexUnknown-6 22 6 -72.73%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6 72 16 -77.78%
benchmark old bytes new bytes delta
BenchmarkMasterIndexLookupSingleIndex-6 160 160 +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6 3200 240 -92.50%
BenchmarkMasterIndexLookupSingleIndexUnknown-6 1232 48 -96.10%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6 4272 128 -97.00%
2018-01-23 22:25:56 -05:00
Matthew Dawson
ebce4b2581
repository/index: Speed up benchmarks and tests
...
When setting up the index used for benchmarking, use math/rand instead of
crypto/rand since the generated ids don't need to be evenly distributed,
and not be secure against guessing. As such, use a different random id
function (only available during tests) that uses math/rand instead.
2018-01-23 22:21:05 -05:00
Igor Fedorenko
953f3d55ee
Optimize pack readHeader() implementation
...
Load pack header length and 15 header entries with single backend
request. This eliminates separate header Load() request for most pack
files and significantly improves index.New() performance.
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-23 22:02:25 -05:00
Alexander Neumann
7e6bfdae79
backend/rest: Implement REST API v2
2018-01-23 23:15:26 +01:00
Alexander Neumann
0f4cbea27d
Document the REST v2 protocol
2018-01-23 23:15:26 +01:00
Alexander Neumann
59782e347c
repository: Fix repack test
...
This reduces the chance of duplicate blobs, otherwise the tests fail
(make the contents of a blob depend on a pseudo-random number instead of
the size, sizes may be duplicate).
2018-01-23 23:14:05 +01:00
Alexander Neumann
e835abeceb
backend/test: Reliably trigger timeout error
2018-01-23 23:14:05 +01:00
Alexander Neumann
8e812b7ac0
errors: Make Cause() unwrap *url.Error
2018-01-23 23:14:05 +01:00
Alexander Neumann
685ce719ad
Add comments for List()
2018-01-23 23:14:05 +01:00
Alexander Neumann
2b39f9f4b2
Update dependencies
...
Among others, this updates minio-go, so that the new "eu-west-3" zone
for AWS is supported.
2018-01-23 19:40:42 +01:00
Alexander Neumann
b63de7c798
script/release: Tag versioned docker image
2018-01-21 21:41:14 +01:00
Alexander Neumann
b0c6e53241
Fix calls to repo/backend.List() everywhere
2018-01-21 21:15:09 +01:00
Alexander Neumann
e9ea268847
Change List() implementation for all backends
2018-01-21 21:15:09 +01:00
Alexander Neumann
dd91b13ff3
Vendor golang.org/x/sync
2018-01-21 18:35:37 +01:00
Alexander Neumann
c4e9d5d11e
backend: Add tests for new List() function
2018-01-21 18:35:37 +01:00
Alexander Neumann
52230b8f07
backend: Rework List()
...
For a discussion see #1567
2018-01-21 18:35:37 +01:00
Alexander Neumann
a3d43a92b3
Merge pull request #1569 from restic/ext-rest-backend-test
...
REST: refactor backend tests
2018-01-21 18:02:09 +01:00
Alexander Neumann
2130897ce0
rest: Add test for external server
2018-01-20 10:25:47 +01:00
Alexander Neumann
67da240068
rest: Refactor backend tests
2018-01-20 10:25:37 +01:00
Alexander Neumann
1046eabf95
rest: Remove unneeded tempdir
2018-01-20 10:13:04 +01:00
Alexander Neumann
663e8284b2
init: Correct text (backend vs. repository)
...
Closes #1565
2018-01-20 09:51:49 +01:00
Alexander Neumann
5a43ca1ccb
Add entry to CHANGELOG
2018-01-20 09:50:31 +01:00
Alexander Neumann
200415e0a1
Merge pull request #1564 from restic/sftp-password-prompt
...
sftp: Prompt for password, don't terminate on SIGINT
2018-01-20 09:48:17 +01:00
Alexander Neumann
ea50687e8b
Merge pull request #1568 from martinseener/patch-1
...
Added hint where to place restic.exe on windows
2018-01-20 09:47:50 +01:00
Alexander Neumann
7335a2492f
Add PR to changelog
2018-01-19 13:05:55 +01:00
Alexander Neumann
bcb6881ffb
Merge pull request #1548 from ifedorenko/checker-backend-Test
...
checker: Optimize checker.Packs()
2018-01-19 13:05:35 +01:00
Alexander Neumann
37df829b3d
CONTRIBUTING: clarify wording
2018-01-19 13:00:49 +01:00
Martin
39f1ba1897
Added hint where to place restic.exe on windows
...
I've added a hint for pre-compiled binaries on windows where to place them for easier use in the shell or in scripts without the hassle for absolute paths.
2018-01-19 12:24:09 +01:00
Igor Fedorenko
231076fa4a
checker: Optimize checker.Packs()
...
Use result of single repository.List() to find both missing and
orphaned data packs. For 500GB repository this eliminates ~100K
repository.Test() calls and improves check time by >30M in my
environment (~45min before this change and ~7min after).
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-18 20:50:39 -05:00
Alexander Neumann
0bdb131521
Remove SuspendSignalHandler
2018-01-17 23:14:47 +01:00
Alexander Neumann
c31a5e7e5c
Add argument to Skipf()
2018-01-17 23:14:37 +01:00
Alexander Neumann
05958caf6e
sftp: Prompt for password, don't terminate on SIGINT
...
This is a follow-up on fb9729fdb9
, which
runs the `ssh` in its own process group and selects that process group
as the foreground group. After the sftp connection is established,
restic switches back to the previous foreground process group.
This allows `ssh` to prompt for the password, but it won't receive
the interrupt signal (SIGINT, ^C) later on, because it is not in the
foreground process group any more, allowing a clean tear down.
2018-01-17 23:02:47 +01:00
Alexander Neumann
87e31799f2
Merge pull request #1483 from dstosberg/master
...
document how to create a full backup without running restic as root
2018-01-16 17:21:55 +01:00
Alexander Neumann
35528506a6
Merge pull request #1554 from restic/fix-988
...
fuse: Correct behavior for reading after EOF, add snapshot template string
2018-01-16 17:17:55 +01:00
Alexander Neumann
2544978aaf
Add entry to changelog
2018-01-15 14:47:17 +01:00
Alexander Neumann
97565d68a2
fuse/mount: Add option for snapshot template
2018-01-15 14:47:17 +01:00
Alexander Neumann
f49f5c5903
fuse: Don't return an error for EOF
2018-01-13 10:14:10 +01:00
Alexander Neumann
d89f2e5226
fuse: Remove debug log message
2018-01-13 10:10:07 +01:00
Alexander Neumann
02f4f5dc66
Update build.go, correct version detection
2018-01-12 22:06:42 +01:00
Alexander Neumann
5723636b35
Add entry for changelog
2018-01-09 21:57:07 +01:00
Alexander Neumann
5632ca4f44
Merge pull request #1538 from MJDSys/make_lookup_internal
...
repository/index: Avoid creating an error when checking if an id is in the index.
2018-01-09 21:55:37 +01:00
Matthew Dawson
539599d1f1
repository/index: Optimize index.Has()
...
When backing up several million files (>14M tested here) with few changes,
a large amount of time is spent failing to find an id in an index and creating
an error to signify this. Since this is checked using the Has method,
which doesn't use this error, this time creating the error is wasted.
Instead, directly check if the given id and type are present in the index.
This also avoids reporting all the packs containing this blob, further
reducing cpu usage.
2018-01-08 21:46:17 +01:00
Alexander Neumann
d77a326bb0
Add benchmark for Index.Has()
2018-01-08 21:10:30 +01:00
Dennis Stosberg
60a7a9d019
manual: document how to create a full backup without running restic as root
2018-01-08 19:59:15 +01:00
Alexander Neumann
99f0fce673
Merge pull request #1536 from restic/fix-restorer-permissions
...
restore: Fix directory permissions
2018-01-07 17:18:29 +01:00
Alexander Neumann
b708cfee3f
Add entry to changelog
2018-01-07 15:16:16 +01:00
Alexander Neumann
b8620429e2
restorer: Set directory mode as last step
...
Closes #1512
2018-01-07 15:13:24 +01:00
Alexander Neumann
aab414b368
Add test case for #1512
2018-01-07 14:50:47 +01:00
Alexander Neumann
e120b17abd
Update PR template
2018-01-07 11:53:30 +01:00
Alexander Neumann
5f43e17918
Merge pull request #1535 from ifedorenko/check-retry
...
fixed restic-check does not retry backend.Test failures
2018-01-07 09:43:49 +01:00
Alexander Neumann
489eef5a6f
Merge pull request #1534 from restic/improve-error-restore
...
restore: Improve error message for more than one ID
2018-01-07 09:40:06 +01:00
Igor Fedorenko
8c550ca011
fixed restic-check does not retry backend.Test failures
...
added missing RetryBackend.Test implementation
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-01-06 23:22:35 -05:00
Alexander Neumann
032621289b
errors: Add "Fatal" prefix for Fatalf()
2018-01-06 23:27:47 +01:00
Alexander Neumann
92316a9853
restore: Improve error message for more than one ID
2018-01-06 22:44:18 +01:00
Alexander Neumann
180741609e
Merge pull request #1530 from juliangehring/patch-1
...
Fix code block formatting in FAQs
2018-01-06 18:46:14 +01:00
Julian Gehring
70250762f3
Fix code block formatting in FAQs
...
Fixes the formatting of a code block in the FAQs and makes it render properly.
2018-01-05 23:02:03 +00:00
Alexander Neumann
39ba17a2d6
Add entry to CHANGELOG
2018-01-05 19:29:02 +01:00
Alexander Neumann
cfe8c8c9cd
Merge pull request #1529 from restic/fix-1528
...
Fix creating missing dirs in local/sftp repo
2018-01-05 19:28:17 +01:00
Alexander Neumann
b45fc89512
local/sftp: Create repo dirs on demand in Save()
2018-01-05 17:51:09 +01:00
Alexander Neumann
aabc0ccaa7
fs: Make HasPathPrefix work with relative paths
2018-01-05 17:38:52 +01:00
Alexander Neumann
2bc4d200d4
Merge pull request #1518 from restic/fix-1317-again
...
forget: Also run prune when only IDs are forgotten
2018-01-02 20:39:12 +01:00
Alexander Neumann
0247fe01c0
Update homepage URL in README
2018-01-01 21:50:44 +01:00
Alexander Neumann
c912b38bf0
forget: Also run prune when only IDs are forgotten
2018-01-01 21:27:40 +01:00
Alexander Neumann
559946c58a
Add linebreak to VERSION generation
2017-12-30 09:55:56 +01:00
Alexander Neumann
a99637c613
Merge pull request #1511 from restic/improve-http-limiting
...
Improve limiting HTTP based backends
2017-12-29 20:57:02 +01:00
Alexander Neumann
36501dda73
Add changelog issue
2017-12-29 12:51:43 +01:00
Alexander Neumann
18ecd9df30
Improve limiting HTTP based backends
2017-12-29 12:43:49 +01:00
Alexander Neumann
c686dd0448
Merge pull request #1507 from restic/fix-fuse-list-timeout
...
fuse: Only reload list of snapshots once per minute
2017-12-28 14:06:59 +01:00
Alexander Neumann
6d91d468e7
Add entry to releases
2017-12-28 13:20:19 +01:00
Alexander Neumann
5f9b5b0219
Add entry to changelog
2017-12-28 13:20:03 +01:00
Alexander Neumann
3f7d85360a
fuse: Only reload list of snapshots once per minute
2017-12-28 13:18:27 +01:00
Alexander Neumann
8d8456590c
Update release script, add check for changelog/releases
2017-12-27 23:32:11 +01:00
Alexander Neumann
85f9f3e290
Fix changelog template for GitHub
2017-12-27 23:26:19 +01:00
Alexander Neumann
9e1180a29b
Fix changelog IDs
2017-12-27 23:22:35 +01:00
Alexander Neumann
e17c1096a0
Add date for 0.8.1
2017-12-27 22:42:15 +01:00
Alexander Neumann
546d6f36b2
Add VERSION for 0.8.1
2017-12-27 22:31:03 +01:00
Alexander Neumann
6ecd14d780
Update manpages and auto-completion
2017-12-27 22:31:03 +01:00
Alexander Neumann
f6ed7dc013
Generate CHANGELOG.md for 0.8.1
2017-12-27 22:30:57 +01:00
Alexander Neumann
e290f2591e
Merge pull request #1504 from restic/changelog-generator
...
Replace manual CHANGELOG with generated one
2017-12-27 22:00:15 +01:00
Alexander Neumann
75f90ca303
Run calens on Travis
2017-12-27 21:21:21 +01:00
Alexander Neumann
ca1430184f
Replace CHANGELOG.md with generated version
2017-12-27 21:07:26 +01:00
Alexander Neumann
a297ab9d7c
Add PrimaryURL to github template
2017-12-27 19:30:04 +01:00
Alexander Neumann
f078525d98
Add changelog entries
2017-12-27 19:22:06 +01:00
Alexander Neumann
e03cc81a9a
Add changelog generation to release script
2017-12-27 19:21:55 +01:00
Alexander Neumann
af27f1dde5
Merge pull request #1503 from gliptak/patch-2
...
Correct golint warning on return
2017-12-27 12:52:31 +01:00
Gábor Lipták
32505c3916
Correct golint warning on return
2017-12-26 17:09:41 -05:00
Alexander Neumann
9a8d5a1bff
Merge pull request #1499 from ametzler/improve-docs-exclude-396
...
doc: Improvements for --exclude
2017-12-24 21:10:28 +01:00
Alexander Neumann
740ee787c1
Merge pull request #1501 from gliptak/patch-1
...
Correct typos
2017-12-24 21:10:08 +01:00
Gábor Lipták
2eba0bfeec
Correct typos
2017-12-24 12:06:52 -05:00
Andreas Metzler
d780ec4bce
doc: Improvements for --exclude
...
Describe exclude pattern handling, adding multiple examples.
Closes #396
2017-12-24 15:39:18 +01:00
Alexander Neumann
6b564d21b3
Merge pull request #1493 from restic/retry-delete
...
backend: Retry deletes
2017-12-23 11:53:09 +01:00
Alexander Neumann
6c2b2a58ad
backend: Retry deletes
2017-12-22 22:41:28 +01:00
Alexander Neumann
b80b68dcb3
doc: Correct setence
2017-12-22 17:59:52 +01:00
Alexander Neumann
29c92ca415
Merge pull request #1491 from restic/improve-b2-config
...
b2: Warn when account ID or key is empty
2017-12-20 18:06:58 +01:00
Alexander Neumann
bc04ce8e6b
Merge pull request #1482 from restic/fix-1478
...
backend: Only return top-level files for most dirs
2017-12-20 18:06:47 +01:00
Alexander Neumann
6b6b75fa4a
Merge pull request #1488 from da2x/patch-1
...
Doc: Specify `key passwd` usage
2017-12-19 21:19:38 +01:00
Alexander Neumann
84e493beba
b2: Warn when account ID or key is empty
...
Closes #1490
2017-12-19 21:12:38 +01:00
Daniel Aleksandersen
323376efa2
Doc: Specify `key passwd` usage
...
Specify that `key passwd` is used to change a password.
2017-12-19 04:53:31 +01:00
Alexander Neumann
e353b00501
Merge pull request #1481 from restic/add-stdin-path
...
backup: Reject directories in filename for --stdin
2017-12-14 19:36:11 +01:00
Alexander Neumann
2510d770ab
Add question about positive things to issue template
2017-12-14 19:22:23 +01:00
Alexander Neumann
7d8765a937
backend: Only return top-level files for most dirs
...
Fixes #1478
2017-12-14 19:14:16 +01:00
Alexander Neumann
81a04656c5
fuse: Cleanup node name
2017-12-13 20:21:18 +01:00
Alexander Neumann
2f26fb8834
Only print cache warning for terminals
2017-12-13 19:58:10 +01:00
Alexander Neumann
d3ebe1311f
backup: Reject filenames with paths read from stdin
2017-12-13 19:55:53 +01:00
Alexander Neumann
42a8c19aae
Merge pull request #1476 from harshavardhana/fix
...
Since upgrade to minio-go 4.0 remove workaround
2017-12-09 11:45:29 +01:00
Harshavardhana
27ccea6371
Since upgrade to minio-go 4.0 remove workaround
...
We added previously a code to fix the issue of chaining
credentials, we do not need this anymore since the
upstream minio-go already has this relevant change.
2017-12-09 02:01:42 -08:00
Alexander Neumann
4f46b4f393
Merge pull request #1475 from restic/update-minio-go
...
Update to minio-go v4
2017-12-09 10:12:05 +01:00
Alexander Neumann
221e741537
Fix broken link in README
2017-12-09 10:04:39 +01:00
Alexander Neumann
8b3b7bc5ef
s3: Use context
2017-12-08 22:04:55 +01:00
Alexander Neumann
934ae1b559
Update to minio-go 4
2017-12-08 21:52:50 +01:00
Alexander Neumann
0e7e3cb714
Update minio-go
2017-12-08 20:45:59 +01:00
Alexander Neumann
95b6e4e9e9
Add entry to CHANGELOG
2017-12-08 20:29:50 +01:00
Alexander Neumann
3a5e040b7e
Merge pull request #1439 from armhold/propagate-context
...
replace ad-hoc context.TODO() with gopts.ctx
2017-12-08 20:27:36 +01:00
Alexander Neumann
28c826868b
Merge pull request #1471 from restic/fix-diff
...
Fix diff
2017-12-06 19:47:17 +01:00
George Armhold
1695c8ed55
use global context for check, debug, dump, find, forget, init, key,
...
list, mount, tag, unlock commands
gh-1434
2017-12-06 07:02:55 -05:00
George Armhold
366622f09a
use global context for cat and ls
...
gh-1434
2017-12-06 05:48:39 -05:00
George Armhold
0dc31c03e1
remove check for context.Canceled
...
gh-1434
2017-12-06 05:38:29 -05:00
Alexander Neumann
0405e67f8b
Change diff output slightly
2017-12-05 22:26:56 +01:00
Alexander Neumann
df350e1f6e
Improve diff
2017-12-05 22:05:22 +01:00
Alexander Neumann
06cb3f7058
Merge pull request #1469 from zertrin/patch-1
...
Update usage help output in doc/manual_rest.rst
2017-12-04 21:36:16 +01:00
Alexander Neumann
56b884be17
Merge pull request #1468 from restic/update-deps
...
Update dependencies
2017-12-04 21:34:52 +01:00
Alexander Neumann
a25d280f3e
Merge pull request #1462 from restic/add-diff
...
Add diff command
2017-12-04 21:34:44 +01:00
zertrin
2253a73837
Update usage help output in doc/manual_rest.rst
...
The usage help in the documentation was outdated relative to current version (0.8.0) with some commands outdated and some missing.
2017-12-04 22:57:00 +08:00
Alexander Neumann
946c8399e2
Update dependenciess
...
Exclude minio-go for now (pin to 3.x.y).
2017-12-03 21:22:33 +01:00
Alexander Neumann
9d0f13c4c0
Prominently mention SIGPIPE issue in CHANGELOG
2017-12-03 19:12:07 +01:00
Alexander Neumann
eb9e2bc79a
manual: Document `diff` command
2017-12-03 18:33:35 +01:00
Alexander Neumann
0722c44ba1
Add entry to CHANGELOG
2017-12-03 18:33:35 +01:00
Alexander Neumann
2424012d75
Add 'diff' command
2017-12-03 18:33:35 +01:00
Alexander Neumann
82ded35706
node: Correctly compare times
2017-12-03 18:33:35 +01:00
Alexander Neumann
69fcb604c8
Merge pull request #1465 from restic/fuse-handle-others
...
fuse: Handle/format all node types correctly
2017-12-03 18:25:49 +01:00
Alexander Neumann
88607fc625
Correctly format all node types
2017-12-03 17:38:55 +01:00
Alexander Neumann
7092af6329
fuse: Handle sockets/fifos/devs correctly
...
Closes #1463
2017-12-03 17:25:00 +01:00
Alexander Neumann
23d7d91597
Merge pull request #1464 from restic/reenable-cache
...
Reenable cache
2017-12-03 16:15:13 +01:00
Alexander Neumann
ad82781743
Reenable cache
...
In 6341c7d72c
, the cache was accidentally
disabled due to a bug, this commit reenables the cache.
2017-12-03 15:52:57 +01:00
George Armhold
20d78ab0d9
add context to repo.Flush() call in test code
...
gh-1434
2017-12-03 07:32:50 -05:00
George Armhold
be24237063
make retry code context-aware.
...
detect cancellation in backend, so that retry code does not keep trying
once user has hit ^c
gh-1434
2017-12-03 07:22:14 -05:00
George Armhold
d886cb5c27
replace ad-hoc context.TODO() with gopts.ctx, so that cancellation
...
can properly trickle down from cmd_*.
gh-1434
2017-12-03 07:22:14 -05:00
Alexander Neumann
63bb1933e5
Merge pull request #1461 from ljani/patch-1
...
Fix a typo
2017-12-03 09:51:08 +01:00
Jani
81e6a9d0d0
Fix a typo
2017-12-03 10:33:46 +02:00
Alexander Neumann
5d4110d2a7
Add entry to CHANGELOG
2017-12-01 21:52:07 +01:00
Alexander Neumann
0cedb3ac9f
Merge pull request #1459 from restic/improve-s3-do-spaces
...
Improve s3 backend for DigitalOcean Spaces
2017-12-01 21:51:30 +01:00
Alexander Neumann
0b44c629f2
retry: Remove file after failed save
2017-11-30 22:05:14 +01:00
Alexander Neumann
2579fe6b7b
Ignore SIGPIPE
...
Handling SIGPIPE made restic abort when a TCP connection was reset by a
server. This happened on DigitalOcean Spaces, which uses the s3 backend.
2017-11-30 21:23:43 +01:00
Alexander Neumann
812ce4bfc4
Add entry to CHANGELOG
2017-11-29 18:44:03 +01:00
Alexander Neumann
410efe0694
Merge pull request #1454 from restic/fix-cache-dir
...
Fix cache dir detection
2017-11-29 18:43:07 +01:00
Alexander Neumann
b2d944d5cb
Merge pull request #1452 from restic/add-with-atime
...
backup: Do not save access time by default
2017-11-29 18:42:45 +01:00
Alexander Neumann
b846c3915c
Add entry to CHANGELOG
2017-11-28 21:41:38 +01:00
Alexander Neumann
ffbc68aa2e
Document access time handling
2017-11-28 21:36:20 +01:00
Alexander Neumann
eddb8549ef
backup: By default, do not save the access time
...
This can be re-enabled with `--with-atime`.
2017-11-28 21:31:35 +01:00
Alexander Neumann
bb44855078
Add pull request template
2017-11-28 19:40:27 +01:00
Alexander Neumann
2567026ccb
Merge pull request #1447 from fawick/fixMultipleRejectIfPresent
...
Fix multiple rejectIfPresent cancelling each other
2017-11-28 19:20:29 +01:00
Alexander Neumann
0cc8fc6f18
Merge pull request #1431 from n0npax/master
...
fix #1411
2017-11-27 21:54:06 +01:00
Alexander Neumann
cc81b916a6
Add entry to CHANGELOG
2017-11-27 21:40:13 +01:00
Fabian Wickborn
27fadd2c6e
Document approach for multiple reject-if-present test
2017-11-27 21:38:15 +01:00
Alexander Neumann
dc38265b54
Merge pull request #1436 from restic/remove-old-cache
...
Remove old cache directories
2017-11-27 21:37:05 +01:00
Fabian Wickborn
1ea518d5ef
cmd/restic: Use a dedicated cache for each rejectIfPresent
2017-11-27 17:33:53 +01:00
Fabian Wickborn
901cd5edef
cmd/restic: Add test for rejectIfPresent bug
...
All RejectFuncs returned by rejectIfPresent share the same rejection
cache and hence might cancel each other out.
2017-11-27 17:26:19 +01:00
Alexander Neumann
e1fd47765b
Fix scripts/release.go
2017-11-26 19:59:21 +01:00
Alexander Neumann
c02923fbfc
Add VERSION for 0.8.0
2017-11-26 19:46:16 +01:00
Alexander Neumann
7c5ce83044
Update manpages and auto-completion
2017-11-26 19:46:16 +01:00
Alexander Neumann
37e2e9a844
Add version to CHANGELOG
2017-11-26 19:46:01 +01:00
Alexander Neumann
26e5db1849
Merge pull request #1446 from restic/fix-relative-restore
...
restore: Fix restore to relative path
2017-11-26 19:03:34 +01:00
Alexander Neumann
a2766ffe0c
CI: return directly after an error occurred
2017-11-26 18:50:40 +01:00
Alexander Neumann
0f5e38609f
restore: Fix restore to relative path
2017-11-26 18:36:48 +01:00
Alexander Neumann
f178cbf93d
Merge pull request #1445 from restic/clean-node-name
...
restorer: Clean node names
2017-11-26 17:34:05 +01:00
Alexander Neumann
c8096ca8d2
Add entry to CHANGELOG
2017-11-26 15:31:09 +01:00
Alexander Neumann
27d29b9853
restorer: Make sure node names are clean
2017-11-26 15:28:45 +01:00
Alexander Neumann
8a171731ba
restorer: Add tests for invalid node names
2017-11-26 15:28:37 +01:00
Alexander Neumann
abde9e2fba
doc: Add --cacert to REST section
2017-11-26 10:09:54 +01:00
Alexander Neumann
6a4a328bbc
Merge pull request #1443 from restic/fix-init-rest-cacert
...
rest: Use client for creating the repository
2017-11-25 22:12:18 +01:00
Alexander Neumann
8253fadc96
doc: Fix typo
2017-11-25 22:11:47 +01:00
Alexander Neumann
134abbd82b
rest: Use client for creating the repository
...
Before, creating a new repo via REST would use the defaut HTTP client,
which is not a problem unless the server uses HTTPS and a TLS
certificate which isn't signed by a CA in the system's CA store. In this
case, all commands work except the 'init' command, which fails with a
message like "invalid certificate".
2017-11-25 20:56:40 +01:00
Alexander Neumann
fe557b022a
Add entry to CHANGELOG
2017-11-25 12:12:08 +01:00
Alexander Neumann
cd8226130a
Improve issue template
2017-11-25 11:43:44 +01:00
Alexander Neumann
1ebf0e8de8
Merge pull request #1437 from restic/fix-1292
...
s3: Document and remove default prefix
2017-11-25 11:34:26 +01:00
Alexander Neumann
37ea764000
cache: Add more documentation
2017-11-25 11:28:59 +01:00
Alexander Neumann
0fdb9a6129
Merge pull request #1426 from pmkane/master
...
fix the signal name for progress reports in the docs
2017-11-24 21:58:11 +01:00
Alexander Neumann
47b326b7b5
Merge pull request #1423 from harshavardhana/creds
...
Fix chaining of credentials for minio-go
2017-11-24 21:57:52 +01:00
Alexander Neumann
e2cf6eb434
Merge pull request #1428 from stephengroat/patch-1
...
use default brew formula
2017-11-24 21:38:04 +01:00
Alexander Neumann
f79698dcdd
Merge pull request #1410 from armhold/deadlock2
...
unify behavior for max http connections across backends
2017-11-24 21:32:56 +01:00
Alexander Neumann
35a5307db3
Merge pull request #1415 from armhold/signals
...
also handle SIGPIPE in cleanup routines
2017-11-24 21:28:08 +01:00
Alexander Neumann
6341c7d72c
cache: Add option to remove old cache dirs
2017-11-24 20:53:28 +01:00
Alexander Neumann
f4bab789b8
cache: Simplify cache dir creation
2017-11-24 20:53:26 +01:00
Alexander Neumann
fa893ee477
cache: Add detection code for old cache dirs
2017-11-24 20:53:26 +01:00
Alexander Neumann
014cec06f1
Add entry to CHANGELOG
2017-11-21 21:33:09 +01:00
Alexander Neumann
431ab5aa6a
manual: Add hint about old default prefix
2017-11-21 21:33:09 +01:00
Alexander Neumann
262b0cd9d4
s3: Remove default prefix "/restic"
2017-11-21 21:33:09 +01:00
Alexander Neumann
e83ec17e95
s3: Correct comment
2017-11-20 22:21:39 +01:00
Alexander Neumann
ea593fca1b
cache: Correctly return dir for Windows/darwin
2017-11-20 06:11:18 +01:00
Alexander Neumann
fe1f151ae1
cache: Return error during default dir detection
2017-11-20 06:10:42 +01:00
n0npax
b12bba4e2a
fix #1411
...
replace panic during index save with fatal error
2017-11-19 11:40:47 +01:00
Stephen
e2005e02bb
use default brew formula
2017-11-18 08:02:41 -08:00
Harshavardhana
41c8c946ba
Fix chaining of credentials for minio-go
...
chaining failed because chaining provider
was only looking for subsequent credentials
provider after an error. Writer a new
chaining provider which proceeds to fetch
new credentials also under situations where
providers do not return but instead return
no keys at all.
Fixes https://github.com/restic/restic/issues/1422
2017-11-18 02:51:12 -08:00
Patrick Kane
fe08686558
fix the signal name for progress reports in the docs
2017-11-16 19:17:25 -08:00
George Armhold
0ed2401711
exit 1 if received signal is other than SIGINT
...
send cleanup msg to stderr, not stdout
gh-1413
2017-11-09 07:16:01 -05:00
Alexander Neumann
06bd606d85
Merge pull request #1414 from bket/1307
...
Fix test failing on OpenBSD #1307
2017-11-08 18:54:44 +01:00
George Armhold
c347431907
also handle SIGPIPE in cleanup routines
...
fixes gh-1413: restic fails to cleanup locks when bash pipeline fails
2017-11-05 07:14:27 -05:00
Björn Ketelaars
f63d7048f9
Fix test failing on OpenBSD #1307
...
Is seems that #1307 is similar to #1087 , which describes a comparable
observation on Apple's new filesystem. #1389 Has been committed and
fixes the problem on Darwin.
Although I'm not sure if the root cause of the issue is the same the
solution is similar for OpenBSD, and leverages #1389 .
2017-11-05 07:27:58 +01:00
Alexander Neumann
f39f7c76dd
Merge pull request #1406 from michaeldorner/master
...
Minor fixes on README and LICENSE
2017-11-04 09:58:21 +01:00
George Armhold
0268d0e7d6
swift backend: limit http concurrency in Save(), Stat(), Test(), Remove(),
...
List().
move comment regarding problematic List() backend api (it's s3's ListObjects
that has a problem, NOT swift's ObjectsWalk).
As per discussion in PR #1399 .
2017-11-02 18:29:32 -04:00
George Armhold
8515d093e0
swift backend: fix premature release of semaphore in Load() & document
...
concurrency issue in List().
refactor wrapReader from b2 -> semaphore so it can be used elsewhere.
As per discussion in PR #1399 .
2017-11-02 12:38:17 -04:00
Michael Dorner
fe3f326d8d
Update LICENSE
...
- improved formatting of license information (BSD 2-clase) so Github can automatically detect it.
2017-11-02 11:39:49 +01:00
Michael Dorner
8170db40c7
Update README.rst
...
- added Codecov badge
- added link to OSI for the license
2017-11-02 11:36:25 +01:00
George Armhold
99ac0da4bc
s3 backend: limit http concurrency in Save(), Stat(), Test(), Remove()
...
NB: List() is NOT currently limited, as it would cause deadlock due to
be.client.ListObjects() implementation.
as per discussion in PR #1399
2017-11-01 09:40:54 -04:00
Alexander Neumann
7e2c93420f
Merge pull request #1397 from restic/crypto-aead
...
crypto: Make crypto.Key implement cipher.AEAD
2017-11-01 13:21:10 +01:00
Alexander Neumann
6d46824fb0
Pass in a nil buffer to Open()
2017-11-01 10:30:40 +01:00
Alexander Neumann
bb435b39d9
crypto: Rework Seal/Open to use sliceForAppend
2017-11-01 10:30:40 +01:00
Alexander Neumann
2a67d7a6c2
crypto: Correct test function names
2017-11-01 10:30:40 +01:00
Alexander Neumann
ba43c8bab5
crypto: Fix nonce test, make it faster
2017-11-01 10:30:40 +01:00
Alexander Neumann
931e6ed2ac
Use Seal/Open everywhere
2017-11-01 10:30:40 +01:00
Alexander Neumann
a5f0e9ab65
Remove custom Encrypt/Decrypt methods
2017-11-01 10:30:40 +01:00
Alexander Neumann
6fc133ad6a
Run tests on Seal/Open
2017-11-01 10:30:40 +01:00
Alexander Neumann
e1b80859f2
Make crypto.Key implement cipher.AEAD
2017-11-01 10:30:39 +01:00
George Armhold
d069ee31b2
GS backend: limit http concurrency in Save(), Stat(), Test(), Remove(), List()
...
as per discussion in PR #1399
2017-10-31 08:01:43 -04:00
George Armhold
981752ade0
Azure backend: limit http concurrency in Stat(), Test(), Remove()
...
as per discussion in PR #1399
2017-10-31 07:32:30 -04:00
Alexander Neumann
d01d07fc0a
Merge pull request #1400 from armhold/deadlock2
...
log unexpected errs from b2 ListCurrentObject()
2017-10-30 18:39:33 +01:00
Alexander Neumann
526aaca6f5
Merge pull request #1389 from wojas/fix-modtime-tests-on-apfs
...
Fix modtime tests on APFS: allow 1us difference
2017-10-29 14:20:40 +01:00
George Armhold
2f8147af59
log unexpected errs from b2 ListCurrentObject()
...
gh-1385
2017-10-29 08:53:39 -04:00
Konrad Wojas
f3016a9096
Darwin test fix: allow 1μs timestamp difference
...
On Darwin, allow a 1μs difference in restored timestamps, because
macOS <10.13 cannot restore with nanosecond precision and the current
version of Go (1.9.2) does not yet support the new syscall required
for this. (#1087 #1389 )
2017-10-29 20:48:07 +08:00
Alexander Neumann
f854a41ba9
Merge pull request #1399 from armhold/deadlock2
...
prevent deadlock in List() for B2 when b2.connections=1
2017-10-29 09:26:46 +01:00
Alexander Neumann
ca3cadef5e
Merge pull request #1398 from armhold/sftp-readdir
...
sftp ReadDir: add path to return error messages (gh-1323)
2017-10-29 09:26:42 +01:00
George Armhold
3304b0fcf0
prevent deadlock in List() for B2 when b2.connections=1
...
This is a fix for the following situation (gh-1188):
List() grabs a semaphore token upon entry, starts a goroutine, and
does not release the token until the routine exits (via a defer).
The goroutine iterates over the results from ListCurrentObjects(),
sending them one at a time to a channel, where they are ultimately
processed by be.Load().
Since be.Load() also needs a token, this will result in deadlock if
b2.connections=1.
This fix changes List() so that the token is only held during the call
to ListCurrentObjects().
2017-10-28 18:46:47 -04:00
George Armhold
d8938e259a
sftp ReadDir: add path to return error messages (gh-1323)
...
fix missing "Close" string in debug log fmt
2017-10-28 14:16:27 -04:00
Alexander Neumann
53a554c89d
Merge pull request #1395 from restic/small-crypto-improvements
...
Small improvements
2017-10-28 12:57:18 +02:00
Alexander Neumann
e71db01230
Improve crypto test
2017-10-28 12:09:33 +02:00
Alexander Neumann
178e946fc7
Rename KDFParams -> Params
2017-10-28 10:28:29 +02:00
Alexander Neumann
f3bff12939
Merge pull request #1394 from zcalusic/master
...
Use lowercase in messages from check/tag commands, too
2017-10-28 09:56:29 +02:00
Alexander Neumann
7a99418dc5
Merge pull request #1393 from armhold/lint-errcheck
...
detect errors from fs.Walk() in local backend List()
2017-10-28 09:56:11 +02:00
Alexander Neumann
c71ba466ea
Merge pull request #1391 from armhold/b2-listmax
...
pass in defaultListMaxItems to b2Backend constructor
2017-10-28 09:54:57 +02:00
Alexander Neumann
8ce5d35543
Merge pull request #1390 from armhold/lint
...
small cleanup
2017-10-28 09:54:35 +02:00
Zlatko Čalušić
134f834c60
Use lowercase in messages from check/tag commands, too
2017-10-27 21:06:34 +02:00
George Armhold
8a37c07295
send errors from fs.Walk() to debug log
...
clarify non-err returns from Walk where err is already proved to be nil
2017-10-27 08:41:17 -04:00
George Armhold
bd0ada7842
go fmt
2017-10-26 16:37:11 -04:00
George Armhold
eea96f652d
go fmt
2017-10-26 16:22:10 -04:00
George Armhold
38c3061df7
pass in defaultListMaxItems to b2Backend constructor
...
gh-1385
2017-10-26 14:22:16 -04:00
George Armhold
f5fa602482
detect and return error from file Close() in Node.createFileAt()
...
gh-1385
2017-10-26 13:53:31 -04:00
Alexander Neumann
e44ac55f63
Merge pull request #1387 from wojas/snapshots-filter-last
...
Add --last flag to snapshots command
2017-10-26 19:08:49 +02:00
Alexander Neumann
f1cfb73a8b
Merge pull request #1384 from wojas/reduce-ticker-refresh
...
Reduce ticker refresh rate with RESTIC_PROGRESS_FPS env
2017-10-26 19:02:59 +02:00
Konrad Wojas
5b96885c6d
Control progress rate with RESTIC_PROGRESS_FPS env
...
Add a RESTIC_PROGRESS_FPS environment variable to limit the interval
at which the progress indicator updates (allowed values: 1-60).
The default rate of 60 FPS can cause high terminal CPU load on some
systems, like iTerm2 on macOS with font anti-aliasing enabled.
Usage:
RESTIC_PROGRESS_FPS=1 restic ...
RESTIC_PROGRESS_FPS=60 restic ...
2017-10-26 14:46:56 +08:00
Konrad Wojas
c5da90a5b7
Add --last flag to snapshots command
...
Add --last flag to snapshots command to only show the last entry for any
(hostname, paths) combination.
This makes it easier to check when various paths were last backed up.
2017-10-26 14:02:29 +08:00
George Armhold
bcdebfb84e
small cleanup:
...
- be explicit when discarding returned errors from .Close(), etc.
- remove named return values from funcs when naked return not used
- fix some "err" shadowing when redeclaration not needed
2017-10-25 12:03:55 -04:00
Alexander Neumann
359b273649
Merge pull request #1381 from tbm/broken-link
...
Remove Markdown syntax from documentation
2017-10-24 20:38:40 +02:00
Martin Michlmayr
2e2c8dc620
Remove Markdown syntax from documentation
...
Convert a link from Markdown to RST syntax. This link must
have been missed during the recent conversion.
2017-10-24 14:40:33 +01:00
Alexander Neumann
8d37b723ca
Upadte vendored dependencies
2017-10-22 10:07:36 +02:00
Alexander Neumann
315b7f282f
Remove explicit version constraints
2017-10-22 10:07:23 +02:00
Alexander Neumann
a3f8e9dfa7
Add entry to CHANGELOG
2017-10-21 12:22:50 +02:00
Alexander Neumann
982810f7cc
Merge pull request #1368 from TobyLL/master
...
Ignore comments (lines starting with #) in the --files-from file
2017-10-21 12:21:11 +02:00
Alexander Neumann
90b96d19cd
Merge pull request #1365 from felix9/fix_1068
...
Fix failure to detect some legacy s3 repos
2017-10-21 12:19:58 +02:00
Alexander Neumann
6a52bb6f54
Merge pull request #1374 from restic/update-blazer
...
Update github.com/kurin/blazer
2017-10-21 11:26:49 +02:00
Alexander Neumann
cacaa4393f
Merge pull request #1373 from restic/check-gopkg-lock
...
CI: Check the vendor dir and Gopkg.lock
2017-10-21 11:14:30 +02:00
Alexander Neumann
d63ab4e9a4
Merge pull request #1358 from prattmic/chunk_size
...
gs: add option to set chunk size
2017-10-21 11:13:48 +02:00
Alexander Neumann
ca6daec8dd
CI: Check the vendor dir and Gopkg.lock
2017-10-21 10:50:38 +02:00
Alexander Neumann
c87f2420a6
Update github.com/kurin/blazer
2017-10-21 10:30:39 +02:00
Alexander Neumann
f5bbbc52f4
Update issue template
2017-10-21 10:17:58 +02:00
TobyLL
9e3dde8ec7
Fix whitespace
2017-10-19 15:52:06 +01:00
TobyLL
9dba182e51
Ignore comments (lines starting with #) in the --files-from file
2017-10-19 15:48:22 +01:00
Felix Lee
944fc857eb
Fix failure to detect some legacy s3 repos
...
Sometimes s3 listobjects for a directory includes an entry for that
directory. The restic s3 backend doesn't expect that and returns
an error.
Symptom is:
ReadDir: invalid key name restic/key/, removing prefix
restic/key/ yielded empty string
I'm not sure when s3 does that; I'm unable to reproduce it myself.
But in any case, it seems correct to ignore that when it happens.
Fixes #1068
2017-10-18 13:45:31 -07:00
Alexander Neumann
7507a658ac
Merge pull request #1362 from restic/fix-backend-retry
...
Fix Save() backend retry
2017-10-18 07:45:56 +02:00
Michael Pratt
9fa4f5eb6b
gs: disable resumable uploads
...
By default, the GCS Go packages have an internal "chunk size" of 8MB,
used for blob uploads.
Media().Do() will buffer a full 8MB from the io.Reader (or less if EOF
is reached) then write that full 8MB to the network all at once.
This behavior does not play nicely with --limit-upload, which only
limits the Reader passed to Media. While the long-term average upload
rate will be correctly limited, the actual network bandwidth will be
very spikey.
e.g., if an 8MB/s connection is limited to 1MB/s, Media().Do() will
spend 8s reading from the rate-limited reader (performing no network
requests), then 1s writing to the network at 8MB/s.
This is bad for network connections hurt by full-speed uploads,
particularly when writing 8MB will take several seconds.
Disable resumable uploads entirely by setting the chunk size to zero.
This causes the io.Reader to be passed further down the request stack,
where there is less (but still some) buffering.
My connection is around 1.5MB/s up, with nominal ~15ms ping times to
8.8.8.8.
Without this change, --limit-upload 1024 results in several seconds of
~200ms ping times (uploading), followed by several seconds of ~15ms ping
times (reading from rate-limited reader). A bandwidth monitor reports
this as several seconds of ~1.5MB/s followed by several seconds of
0.0MB/s.
With this change, --limit-upload 1024 results in ~20ms ping times and
the bandwidth monitor reports a constant ~1MB/s.
I've elected to make this change unconditional of --limit-upload because
the resumable uploads shouldn't be providing much benefit anyways, as
restic already uploads mostly small blobs and already has a retry
mechanism.
--limit-download is not affected by this problem, as Get().Download()
returns the real http.Response.Body without any internal buffering.
Updates #1216
2017-10-17 21:12:04 -07:00
Alexander Neumann
ce4d71d626
backend: Add partial read failure to error backend
2017-10-17 22:11:38 +02:00
Alexander Neumann
8e2ef3f38b
cache: Store first, then store in the backend
...
Store the file in the backend, then rewind the reader and store it
in the cache afterwards.
2017-10-17 22:01:20 +02:00
Alexander Neumann
8dc952775e
backend: Correctly retry Save() calls
...
Make sure the given reader is an io.Seeker and rewind it properly each
time.
2017-10-17 21:46:38 +02:00
Alexander Neumann
99b6163e27
Add 'dump' to manual
2017-10-16 20:24:47 +02:00
Alexander Neumann
beaf55f1fc
dump: Allow absolute paths
2017-10-16 20:22:01 +02:00
Alexander Neumann
980bb9059f
Add entry to changelog
2017-10-16 20:15:19 +02:00
Alexander Neumann
0e7281eb71
Merge pull request #1346 from fawick/cmdCatFile
...
Add dump sub-command for dumping files to stdout
2017-10-16 20:13:44 +02:00
Alexander Neumann
0b6133d7b5
Merge pull request #1360 from mungomat/fuse_bugfix
...
fuse: bugfix: ignore empty tags
2017-10-16 20:07:05 +02:00
Tobias Klein
b57ca64275
fuse: bugfix: ignore empty tags
2017-10-16 16:22:08 +02:00
Alexander Neumann
faadbd734b
Add entry to CHANGELOG
2017-10-14 20:29:47 +02:00
Alexander Neumann
88b0a93409
Merge pull request #1353 from restic/backend-retry
...
Retry failed backend requests
2017-10-14 20:28:57 +02:00
Alexander Neumann
4a995105a9
sftp: Fix Delete()
2017-10-14 16:08:15 +02:00
Alexander Neumann
7fe496f983
Ensure TestDelete runs last
2017-10-14 16:04:29 +02:00
Alexander Neumann
e56370eb5b
Remove Deleter interface
2017-10-14 16:04:29 +02:00
Alexander Neumann
b8af7f63a0
backend test: Always remove files for TestList
2017-10-14 15:56:25 +02:00
Alexander Neumann
3eea555155
Add dependency cenkalti/backoff
2017-10-14 15:56:25 +02:00
Alexander Neumann
897c923cc9
Retry failed backend requests
2017-10-14 15:56:25 +02:00
Alexander Neumann
67193e3deb
Improve error message when creating lock failed
2017-10-14 15:56:25 +02:00
Alexander Neumann
0e722efb09
backend: Add Delete() to restic.Backend interface
2017-10-14 15:56:25 +02:00
Alexander Neumann
3736f33ebf
Merge pull request #1352 from tyll/manual_links
...
Fix links to manual
2017-10-14 14:54:16 +02:00
Fabian Wickborn
d1d9c3f9d7
Renamed cmd_catfile.go to cmd_dump.go
2017-10-14 13:55:21 +02:00
Fabian Wickborn
cd5cbe0910
Rename debug dump related variable and run function
2017-10-14 13:55:00 +02:00
Fabian Wickborn
814e992c0b
Rename subcommand catfile to dump
2017-10-14 11:34:04 +02:00
Till Maas
660fe78735
Fix links to manual
2017-10-13 20:01:19 +02:00
Fabian Wickborn
87d084e18c
Add subcommand dump
2017-10-12 20:24:34 +02:00
Alexander Neumann
9ce2a73fc5
Add issue and PR to the changelog entry
2017-10-12 20:20:06 +02:00
Alexander Neumann
f2314b26ba
Move 'dump' to 'debug dump'
2017-10-12 20:18:45 +02:00
Alexander Neumann
74dcf41f25
Disable 'dump' for non-debug builds
2017-10-12 20:14:48 +02:00
Alexander Neumann
b6ba30186f
Merge pull request #1336 from rmdashrf/rate-limit-backup
...
Add basic rate limiting to backup
2017-10-12 20:08:55 +02:00
rmdashrf
32637a0328
Basic rate limiting implementation.
...
Added `--limit-upload` and `--limit-download` flags to rate limit
backups and restores.
2017-10-11 20:01:20 -07:00
Alexander Neumann
0addd90e14
Merge pull request #1345 from harshavardhana/creds
...
Refactor credentials management to support multiple mechanisms.
2017-10-11 21:01:19 +02:00
Alexander Neumann
1b5ee5b10a
Merge pull request #1343 from mungomat/fuse_speedup
...
fuse mount: speedup
2017-10-11 20:55:52 +02:00
Harshavardhana
042adeb5d0
Refactor credentials management to support multiple mechanisms.
...
This PR adds the ability of chaining the credentials provider,
such that restic as a tool attempts to honor credentials from
multiple different ways.
Currently supported mechanisms are
- static (user-provided)
- IAM profile (only valid inside configured ec2 instances)
- Standard AWS envs (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
- Standard Minio envs (MINIO_ACCESS_KEY, MINIO_SECRET_KEY)
Refer https://github.com/restic/restic/issues/1341
2017-10-09 12:51:39 -07:00
Tobias Klein
7e4ce0dacc
fuse mount: speedup
2017-10-09 10:26:56 +02:00
Alexander Neumann
8ceb22fe8a
Add entry to CHANGELOG
2017-10-08 09:48:22 +02:00
Alexander Neumann
c5553ec855
Merge pull request #1276 from fawick/supply_ca_cert
...
Add REST backend option to use CA root certificate
2017-10-08 09:47:23 +02:00
Alexander Neumann
bb3ed54291
Merge pull request #1337 from mungomat/fuse_updateRepo
...
fuse: mount and backup in parallel (#1330 )
2017-10-08 09:45:28 +02:00
Alexander Neumann
513ba3b6f7
Merge pull request #1340 from jniggemann/fix-664
...
doc: Add info on docker container
2017-10-08 09:44:34 +02:00
Alexander Neumann
17d688afef
Merge pull request #1339 from felix9/fix_1251
...
fixes #1251 , race when writing indexes
2017-10-07 16:36:16 +02:00
Jan Niggemann
d81eee26b3
doc: Add info on docker container
...
fixes #664
2017-10-07 16:03:08 +02:00
Felix Lee
cc5ada63a4
fixes #1251 , race when writing indexes
2017-10-07 05:11:42 -07:00
Alexander Neumann
88fb60e0b5
Improve issue template
2017-10-07 12:16:46 +02:00
Tobias Klein
02200acad0
fuse: mount and backup in parallel ( #1330 )
2017-10-07 11:48:48 +02:00
Alexander Neumann
1a2d190bdb
Merge pull request #1334 from felix9/backup_tags
...
backup should use latest parent regardless of tags
2017-10-07 10:45:02 +02:00
Alexander Neumann
2db4ff168a
Merge pull request #1327 from fawick/fix-1294
...
Create missing lock dir when saving lock
2017-10-07 10:40:29 +02:00
Alexander Neumann
a77c8cc5d2
Add entry to CHANGELOG
2017-10-07 10:23:56 +02:00
Alexander Neumann
b8866c1fe4
Merge pull request #1326 from fawick/rejectionCache
...
Cache evaluated directories in isExcludedByFile
2017-10-07 09:57:54 +02:00
Fabian Wickborn
f0f17db847
Cache evaluated directories in isExcludedByFile
...
Fixes #1271 .
2017-10-06 22:11:22 +02:00
Alexander Neumann
a5c003acb0
Merge pull request #1333 from restic/fix-1328
...
Do not cache invalid/truncated files
2017-10-05 21:36:52 +02:00
Felix Lee
7b44fd0f9d
fix #1143
...
Backup was choosing a parent snapshot that had the same tags, which
makes backup unnecessarily slow when there are newer snapshots with
different tags.
There's no reason parent has to have the same tags.
This change makes backup choose the newest snapshot instead.
2017-10-05 11:48:26 -07:00
Alexander Neumann
cebee0b8fa
cache: Refuse to cache truncated files
2017-10-05 20:40:02 +02:00
Alexander Neumann
d886bc6c48
Ignore invalid index files, print warning
2017-10-05 20:39:53 +02:00
Alexander Neumann
d81adcfaa5
cache: Add file name to error message
2017-10-05 19:30:56 +02:00
Fabian Wickborn
6da9bfbbce
Create missing lock dir when saving lock
2017-10-05 00:07:48 +02:00
Fabian Wickborn
69a6e622d0
Add REST backend option to use CA root certificate
...
Closes #1114 .
2017-10-04 22:14:10 +02:00
Alexander Neumann
1dcfd64028
Merge pull request #1325 from antonlindstrom/lookup-group
...
Add group name in fillUser
2017-10-04 21:32:46 +02:00
Alexander Neumann
5d1c1f721e
Merge pull request #1324 from antonlindstrom/password-feedback
...
Add password successful feedback
2017-10-04 21:31:24 +02:00
Alexander Neumann
fbc8bbf305
Merge pull request #1321 from damienstanton/master
...
fix broken installation link in README
2017-10-04 21:02:33 +02:00
Alexander Neumann
cdef55bb88
Merge pull request #1312 from hgfischer/issue-1119
...
nit: please remove dot imports #1119
2017-10-04 21:01:40 +02:00
Anton Lindstrom
26df48b2aa
Add group name in fillUser
...
This does a lookup of the group name from the GID and adds it to the
Node.
2017-10-04 20:11:54 +02:00
Anton Lindstrom
a7baea0522
Output password successful on terminal stdout
...
This removes the conditions that checks if the password is supplied
through environment variable or file and outputs password is successful
on terminal and when --quiet is not supplied.
2017-10-04 14:55:04 +02:00
Anton Lindstrom
55e6003749
Add password successful feedback
...
This adds some feedback when entering the password on the command line.
When the password is entered and supplied through stdin (and stdout is a
terminal) then the a message saying `password is correct` if correct is
printed.
2017-10-04 13:45:05 +02:00
damienstanton
846acd5d4c
use https
2017-10-03 14:28:17 -04:00
Herbert
43f8145858
Revert "Add .vscode to gitignore"
...
This reverts commit 1b5242b4f9
.
2017-10-03 20:21:54 +02:00
damienstanton
79759928f6
fix broken installation link in README
2017-10-03 13:28:33 -04:00
Alexander Neumann
eb59d28154
Add entry to CHANGELOG
2017-10-03 18:54:07 +02:00
Alexander Neumann
79f63a2e74
Merge pull request #1320 from restic/add-snapshots-json-short-id
...
snapshots: Add short ID to JSON output
2017-10-03 18:53:17 +02:00
Alexander Neumann
6a62254048
Merge pull request #1299 from mungomat/fuse_addSnapshotIDs
...
fuse: Add a snapshot IDs directory (#1102 )
2017-10-03 18:53:05 +02:00
Alexander Neumann
657a1d75af
snapshots: Add short ID to JSON output
...
We're using the short ID in all output to users, so it should also be
included in the JSON output of the `snapshots` command.
2017-10-03 15:24:09 +02:00
Alexander Neumann
2694def56a
Add entry to CHANGELOG
2017-10-03 14:52:10 +02:00
Alexander Neumann
7843341da3
Merge pull request #1319 from antonlindstrom/check-ok-output
...
Add explicit OK output to check command
2017-10-03 14:51:31 +02:00
Anton Lindstrom
d46314648e
Add explicit OK output to check command
...
This adds additional output to the check command when no errors were
found. It means that when all checks have been completed, the following
output is displayed:
No errors were found
The output is added to make sure that it is easier to understand that no
errors were found.
Full example output:
Create exclusive lock for repository
Load indexes
Check all packs
Check snapshots, trees and blobs
No errors were found
2017-10-03 12:57:51 +02:00
Alexander Neumann
e45011af57
Merge pull request #1304 from jannickfahlbusch/showAmountOfSnapshots
...
Show the number of snapshots
2017-10-03 12:41:47 +02:00
Alexander Neumann
fb09884893
Merge pull request #1311 from felix9/dump-no-trees
...
"dump" command no longer has a "trees" option
2017-10-03 12:40:24 +02:00
Alexander Neumann
d1eecafa63
Merge pull request #1314 from jannickfahlbusch/fixTypo
...
Fix typo in comment
2017-10-03 12:39:24 +02:00
Alexander Neumann
b47d991f56
Merge pull request #1316 from restic/prune-wrong-data
...
prune: Warn about wrong plaintext blob ID
2017-10-03 12:39:04 +02:00
Alexander Neumann
553ea812a7
Add entry to CHANGELOG
2017-10-03 12:38:00 +02:00
Alexander Neumann
216e374310
Merge pull request #1315 from prattmic/cache_doc
...
Clarify cache location documentation
2017-10-03 12:36:53 +02:00
Alexander Neumann
034b0b8040
forget: Run prune for manually forgotten snapshots
2017-10-03 11:56:13 +02:00
Michael Pratt
5ab9e12b46
Clarify cache location documentation
...
PR #1287 changed the default cache location on darwin and windows.
Update the changelog and manual to reflect the new behavior.
Since the cache hasn't been included in an official release yet, I've
just changed the main cache changelog entry.
Fixes #1309
2017-10-03 11:44:09 +02:00
Alexander Neumann
abe6e0d22d
Merge pull request #1300 from jniggemann/refactor-manual
...
doc: Refactors the documentation
2017-10-03 11:41:00 +02:00
Jan Niggemann
f5b550191c
doc: Refactors the documentation
...
This commit refactors the documentation according to my proposal in #1273
and the discussion I had with fd0 on IRC.
The bits from the manual that I could not immediately put into the new
structure are contained in manual_rest.rst Anything else is still there,
nothing has been deleted.
I changed the heading markup to follow the convention used in Python’s
Style Guide for documentation, this convention is explained in a comment
at the top of every file.
I also added a paragraph on installing restic on Debian.
2017-10-03 11:21:53 +02:00
Jannick Fahlbusch
3dcacb3730
Fix typo in comment
2017-10-02 18:19:22 +02:00
Herbert
033589a66b
Use rtest on these as well to keep codebase consistent
2017-10-02 17:48:08 +02:00
Jannick Fahlbusch
e46a647c45
Just print the number of snapshots
2017-10-02 17:05:59 +02:00
Alexander Neumann
f26492fc2d
prune: Warn about wrong plaintext blob ID
2017-10-02 16:27:08 +02:00
Herbert
3473c3f7b6
Remove all dot-imports
2017-10-02 15:06:39 +02:00
Herbert
1b5242b4f9
Add .vscode to gitignore
2017-10-02 15:06:23 +02:00
Felix Lee
6d897def1b
also fix man page
2017-10-01 18:33:41 -07:00
Jannick Fahlbusch
2f10e25738
Show the amount of snapshots
2017-10-01 17:14:54 +02:00
Felix Lee
555bd257bd
dump command no longer has 'trees' option
2017-10-01 08:04:52 -07:00
Alexander Neumann
3afd974dea
Add entry to CHANGELOG
2017-10-01 10:35:15 +02:00
Alexander Neumann
f4120c9d45
Merge pull request #1301 from restic/update-blazer
...
Update vendored dependencies
2017-10-01 10:33:54 +02:00
Alexander Neumann
61cb1cc6f8
Update vendored dependencies
...
This includes github.com/kurin/blazer 0.2.0, which resolves #1291
2017-10-01 10:13:39 +02:00
Tobias Klein
49bc1d0b3b
fuse: Add a snapshot IDs directory ( #1102 )
2017-09-30 16:22:14 +02:00
Alexander Neumann
ba23d24dd1
Merge pull request #1298 from restic/fix-1288
...
sftp: Handle/Document tilde character
2017-09-30 12:28:09 +02:00
Alexander Neumann
556a63de19
sftp: Return error when path starts with a tilde (~)
2017-09-30 10:34:23 +02:00
Alexander Neumann
fae3c4d437
faq: Add entry about Synology NAS' sftp path
2017-09-30 10:30:21 +02:00
Alexander Neumann
89c2ed2a1c
manual: Document that sftp does not expand tilde
2017-09-30 10:24:28 +02:00
Alexander Neumann
23f1cb06d6
Fix FAQ
2017-09-30 10:21:56 +02:00
Alexander Neumann
ac92e2dd2d
Add to CHANGELOG entry
2017-09-29 21:45:37 +02:00
Alexander Neumann
bf58425351
Merge pull request #1287 from prattmic/cachedir
...
cache: OS-specific cache directories
2017-09-29 21:42:34 +02:00
Alexander Neumann
a3dc0ab398
Merge pull request #1295 from bachp/patch-2
...
Fix wrong quotes for command in installation.
2017-09-29 21:41:13 +02:00
Pascal Bach
224ebdb8b9
Fix wrong quotes for command in installation.
2017-09-29 10:08:24 +02:00
Alexander Neumann
cf80d295f3
Merge pull request #1285 from bachp/patch-1
...
Add instruction on how to install restic using Nix
2017-09-28 22:03:19 +02:00
Michael Pratt
2133869127
cache: OS-specific cache directories
...
Windows, and to a lesser extent OS X, don't conform to XDG and have
their own preferred locations for caches.
On Windows, use %LOCALAPPDATA%/restic (i.e., ~/AppData/Local/restic). I
can't find authoritative documentation from Microsoft recommending
specifically which of %APPDATA%, %LOCALAPPDATA%, and %TEMP% should be
used for caches, but %LOCALAPPDATA% is where browsers store their
caches, so it seems like a good fit.
On OS X, use ~/Library/Caches/restic, which is recommended by the Apple
documentation. They do suggest using the application "bundle identifier"
as the base folder name, but restic doesn't have one, so I just used
"restic".
2017-09-27 21:16:22 -07:00
Pascal Bach
97330ac621
Add instruction on how to install restic using Nix
2017-09-27 22:41:20 +02:00
Alexander Neumann
1ee1559506
Add release script
2017-09-26 22:14:58 +02:00
Alexander Neumann
eccc336319
Add entry to CHANGELOG
2017-09-26 14:18:37 +02:00
Alexander Neumann
7fe657ec71
Merge pull request #1282 from restic/rework-autogeneration
...
Rework generation of manpages and completion files
2017-09-26 14:16:41 +02:00
Alexander Neumann
77c07bfd19
Remove integration test for manpage
2017-09-26 13:16:55 +02:00
Alexander Neumann
4de938d97a
Update manpages and auto-completion
2017-09-26 13:16:08 +02:00
Alexander Neumann
dad1c87afe
Rework generation of manpages and completion files
...
This commit removes the `manpages` and `autocomplet` commands and
replaces them with the more generic `generate` command. Also, zsh
completion file support was added.
2017-09-26 13:12:12 +02:00
Alexander Neumann
801dbb6d03
Merge pull request #1281 from prattmic/gcs_perms
...
gs: allow backend creation without storage.buckets.get
2017-09-26 09:38:33 +02:00
Michael Pratt
fa0be82da8
gs: allow backend creation without storage.buckets.get
...
If the service account used with restic does not have the
storage.buckets.get permission (in the "Storage Admin" role), Create
cannot use Get to determine if the bucket is accessible.
Rather than always trying to create the bucket on Get error, gracefully
fall back to assuming the bucket is accessible. If it is, restic init
will complete successfully. If it is not, it will fail on a later call.
Here is what init looks like now in different cases.
Service account without "Storage Admin":
Bucket exists and is accessible (this is the case that didn't work
before):
$ ./restic init -r gs:this-bucket-does-exist:/
enter password for new backend:
enter password again:
created restic backend c02e2edb67 at gs:this-bucket-does-exist:/
Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
Bucket exists but is not accessible:
$ ./restic init -r gs:this-bucket-does-exist:/
enter password for new backend:
enter password again:
create key in backend at gs:this-bucket-does-exist:/ failed:
service.Objects.Insert: googleapi: Error 403:
my-service-account@myproject.iam.gserviceaccount.com does not have
storage.objects.create access to object this-bucket-exists/keys/0fa714e695c8ecd58cb467cdeb04d36f3b710f883496a90f23cae0315daf0b93., forbidden
Bucket does not exist:
$ ./restic init -r gs:this-bucket-does-not-exist:/
create backend at gs:this-bucket-does-not-exist:/ failed:
service.Buckets.Insert: googleapi: Error 403:
my-service-account@myproject.iam.gserviceaccount.com does not have storage.buckets.create access to bucket this-bucket-does-not-exist., forbidden
Service account with "Storage Admin":
Bucket exists and is accessible: Same
Bucket exists but is not accessible: Same. Previously this would fail
when Create tried to create the bucket. Now it fails when trying to
create the keys.
Bucket does not exist:
$ ./restic init -r gs:this-bucket-does-not-exist:/
enter password for new backend:
enter password again:
created restic backend c3c48b481d at gs:this-bucket-does-not-exist:/
Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
2017-09-25 22:25:51 -07:00
Alexander Neumann
7e8bc8d362
Merge pull request #1279 from restic/fix-eof-error
...
cache: Synchronize downloading
2017-09-25 16:39:20 +02:00
Alexander Neumann
0bb2a8e0d0
cache: Synchronize downloading
...
This commit adds code to synchronize downloading files to the cache.
Before, requests that came in for files currently downloading would fail
because the file was not completed in the cache. Now, the code waits
until the download is completed.
Closes #1278
2017-09-25 15:58:20 +02:00
Alexander Neumann
2e72b57f2f
Correct debug message
2017-09-25 14:35:37 +02:00
Alexander Neumann
bff1039e3a
Add entry to CHANGELOG
2017-09-25 13:17:44 +02:00
Alexander Neumann
5a999cb77f
Merge pull request #1040 from restic/add-cache
...
Add local cache
2017-09-25 13:13:07 +02:00
Alexander Neumann
3a2539e0ac
doc: Update manpages
2017-09-24 23:13:04 +02:00
Alexander Neumann
e262f35d0a
cache: Auto-remove invalid files
2017-09-24 23:11:47 +02:00
Alexander Neumann
176bfa6529
backend: Improve ReadAt
2017-09-24 23:11:23 +02:00
Alexander Neumann
240c4cf2fd
cache: In case of an error, fall back backend
2017-09-24 23:11:23 +02:00
Alexander Neumann
db5ec5d876
repo: Automatically cache tree-only pack files
2017-09-24 23:11:23 +02:00
Alexander Neumann
e1dfaf5d87
cache: Allow proactive caching of tree packs
...
This commit adds a function to the cache which can decide to proactively
load the complete pack file and store it in the cache. This is helpful
for pack files containing only tree blobs, as it is likely that the same
file is accessed again in the future.
2017-09-24 23:11:23 +02:00
Alexander Neumann
5436154f0d
cache: Add PerformReadahead
2017-09-24 23:11:23 +02:00
Alexander Neumann
809e218d20
cache: Improve debug logs
2017-09-24 23:11:23 +02:00
Alexander Neumann
1eaad6cebb
index: Add TreePacks()
2017-09-24 21:54:53 +02:00
Alexander Neumann
56fccecd06
prune: Repack mixed pack files
2017-09-24 21:54:53 +02:00
Alexander Neumann
3890a947ca
Clear data files in cache
2017-09-24 21:54:53 +02:00
Alexander Neumann
e299272378
repo: Try cached pack files first
2017-09-24 21:54:53 +02:00
Alexander Neumann
70248bd05a
repo: Clear indexes
2017-09-24 21:54:53 +02:00
Alexander Neumann
7a5fde8f5a
repository: Save pack files for trees in cache
2017-09-24 21:54:53 +02:00
Alexander Neumann
62ba9f1950
check: Disable cache by default
2017-09-24 21:54:53 +02:00
Alexander Neumann
610b676444
Automatically exclude current restic cache
2017-09-24 21:54:53 +02:00
Alexander Neumann
58699e3c90
Write CACHEDIR.TAG to cache base directory
2017-09-24 21:54:53 +02:00
Alexander Neumann
9be24a1c9f
Add cache
...
This commits adds rudimentary support for a cache directory, enabled by
default. The cache directory is created if it does not exist. The cache
is used if there's anything in it, newly created snapshot and index
files are written to the cache automatically.
2017-09-24 21:54:53 +02:00
Alexander Neumann
5ace41471e
Merge pull request #1277 from prattmic/gcs_cleanup
...
Document GCS permission requirements
2017-09-24 20:49:02 +02:00
Michael Pratt
3b2106ed30
gs: document required permissions
...
In the manual, state which standard roles the service account must
have to work correctly, as well as the specific permissions required,
for creating even more specific custom roles.
2017-09-24 11:25:57 -07:00
Michael Pratt
5f4f997126
gs: minor comment cleanups
...
* Remove a reference to S3.
* Config can only be used for GCS, not other "gcs compatibile servers".
* Make comments complete sentences.
2017-09-24 10:10:56 -07:00
Alexander Neumann
49d397a419
Merge pull request #1275 from fawick/sort_snapshots
...
Always sort snapshots lists ascending by timestamp
2017-09-24 16:32:40 +02:00
Fabian Wickborn
ea1ab96749
Always sort snapshots lists ascending by timestamp
...
Fixes #1219 .
2017-09-24 13:01:13 +02:00
Alexander Neumann
24c62e719a
Add entry to CHANGELOG
2017-09-23 22:15:10 +02:00
Alexander Neumann
9c6b7f688e
Merge pull request #1270 from restic/sftp-allow-password-prompt
...
sftp: Allow password entry
2017-09-23 22:13:04 +02:00
Alexander Neumann
d41dce5ecb
Merge pull request #1272 from jniggemann/doc-faq-add-prio
...
doc: FAQ: Add info on IO and CPU prioritization
2017-09-23 20:08:17 +02:00
Jan Niggemann
52a3eafede
doc: FAQ: Add info on IO and CPU prioritization
2017-09-23 19:32:07 +02:00
Alexander Neumann
55dfc85159
manual: Add hint for RESTIC_PASSWORD_FILE
2017-09-23 19:16:07 +02:00
Alexander Neumann
a7a478a19e
doc: Correct FAQ
2017-09-23 19:15:21 +02:00
Alexander Neumann
2080afd9de
Merge pull request #1259 from jniggemann/doc-add-restic-check-advice
...
adds advice to run restic check regularly
2017-09-23 14:07:55 +02:00
Alexander Neumann
9aa136b982
Merge pull request #1260 from jniggemann/doc-add-info-on-special-items
...
Doc add info on special items
2017-09-23 14:07:20 +02:00
Alexander Neumann
3a191f37cb
Add entry to CHANGELOG
2017-09-23 14:05:55 +02:00
Alexander Neumann
429106340f
Merge pull request #1267 from harshavardhana/possible-fix-memory
...
Implement Size() and Len() to know the optimal size.
2017-09-23 14:04:15 +02:00
Alexander Neumann
530c73b457
Merge pull request #1269 from mrzv/forget-compact
...
Add --compact option to forget
2017-09-23 14:02:34 +02:00
Alexander Neumann
fb9729fdb9
sftp: Allow password entry
...
This was a bit tricky: We start the ssh binary, but we want it to ignore
SIGINT. In contrast, restic itself should process SIGINT and clean up
properly. Before, we used `setsid()` to give the ssh process its own
process group, but that means it cannot prompt the user for a password
because the tty is gone.
So, now we're passing in two functions that ignore SIGINT just before
the ssh process is started and re-install it after start.
2017-09-23 11:43:33 +02:00
Alexander Neumann
45a09c76ff
Allow suspending SIGINT handler
2017-09-23 11:12:56 +02:00
Dmitriy Morozov
efd65a1b65
Update manpage for forget
2017-09-22 16:35:58 -07:00
Dmitriy Morozov
ae60188eb9
Add --compact option to forget
2017-09-22 16:32:59 -07:00
Jan Niggemann
3b904525d9
manual: Add info on special items (device files)
2017-09-22 22:58:26 +02:00
Jan Niggemann
1e31f5202f
manual: Add info on special items
...
Add info about handling of symlinks and bind-mounts.
Closes : #1014
2017-09-22 22:51:31 +02:00
Jan Niggemann
f12d41138a
Add advice to run check regularly
2017-09-22 22:27:10 +02:00
Harshavardhana
98369f6a5d
Implement Size() and Len() to know the optimal size.
2017-09-22 12:09:17 -07:00
Alexander Neumann
8f9bf1995b
Merge pull request #1265 from restic/improve-packers
...
Improve packers, prepare for cache
2017-09-22 16:16:10 +02:00
Alexander Neumann
e7de3b5f9d
Merge pull request #1266 from JaCoB1123/document_hostname_option
...
Add note about rescan to hostname flag (fixes #1221 )
2017-09-22 15:40:24 +02:00
Alexander Neumann
3541d06d07
repo: Split packers for tree and data
...
The code now bundles tree blobs and data blobs into different pack
files, so we'll end up with pack files that either only contain data or
trees. This is in preparation to adding a cache (#1040 ), because
tree-only pack files can easily be cached later on.
2017-09-22 15:36:47 +02:00
Alexander Neumann
db0e3cd772
repo: Remove packer limits
...
This commit simplifies finding a packer: The first open packer is taken,
and the upper limit for the pack file is removed.
2017-09-22 15:36:47 +02:00
Alexander Neumann
d3fee08f9a
Merge pull request #1263 from restic/cleanups
...
Small cleanups
2017-09-22 15:36:13 +02:00
Jan Bader
727fb6eabe
Add note about rescan to hostname flag ( fixes #1221 )
2017-09-22 14:29:04 +02:00
Alexander Neumann
d610c60991
repo: Remove unused sync.Pool
2017-09-22 12:37:10 +02:00
Alexander Neumann
3f6e11d26e
Allow sorting nodes in trees
2017-09-22 12:37:05 +02:00
Alexander Neumann
a4577769ae
Merge pull request #1262 from mafgh/bytes
...
fix duplicate bytes in prune output
2017-09-22 11:59:10 +02:00
Stefan Völkel
7f927d4774
fix duplicate bytes in prune output
2017-09-22 10:07:24 +02:00
Alexander Neumann
e091673f8f
Merge pull request #1258 from restic/sftp-remove-stat
...
local/sftp: Remove unneeded stat() call
2017-09-21 22:37:12 +02:00
Alexander Neumann
9842eff887
local/sftp: Remove unneeded stat() call
2017-09-21 21:47:03 +02:00
Alexander Neumann
c40b3d3983
Add entry to CHANGELOG
2017-09-21 20:34:17 +02:00
Alexander Neumann
ac5eefdee4
Merge pull request #1249 from mungomat/fuse_symlink
...
fuse: added symlink 'latest' to snapshots-dir
2017-09-21 20:33:10 +02:00
Alexander Neumann
bf508643a5
Merge pull request #1257 from restic/update-deps
...
Update vendored dependencies
2017-09-21 20:19:25 +02:00
Alexander Neumann
02fc16e97d
Update vendored dependencies
2017-09-21 17:48:45 +02:00
Tobias Klein
1a83a739dc
fuse: added symlink 'latest' to snapshots-dir
2017-09-21 16:41:20 +02:00
Alexander Neumann
81473f4538
Add VERSION file for 0.7.3
2017-09-20 20:50:07 +02:00
Alexander Neumann
e1a847e4d1
Add new version to CHANGELOG
2017-09-20 20:49:55 +02:00
Alexander Neumann
0f426c3795
Merge pull request #1254 from jniggemann/doc_metadata
...
documents metadata handling, fixes #647
2017-09-20 20:24:57 +02:00
Alexander Neumann
6df3d169b8
Add entry to CHANGELOG
2017-09-20 11:05:35 +02:00
Alexander Neumann
5479daa6d4
Merge pull request #1247 from restic/fix-1246
...
Fix backend List()
2017-09-20 11:04:05 +02:00
Jan Niggemann
397fec0152
documents metadata handling, fixes #647
2017-09-19 15:49:54 +02:00
Alexander Neumann
d7e644272f
prune: Add plausibility check
2017-09-19 10:50:07 +02:00
Alexander Neumann
e91749bbb0
Merge pull request #1245 from anarcat/faq
...
add explanation of restic automation
2017-09-19 10:46:55 +02:00
Antoine Beaupré
bcd1e45ba7
fix typo, add note about file permissions
2017-09-18 08:55:18 -04:00
Alexander Neumann
4c6b626db6
backend: Improve TestList
2017-09-18 13:18:42 +02:00
Alexander Neumann
835ba16c27
b2: Add pagination for List()
2017-09-18 12:13:35 +02:00
Alexander Neumann
3b6a580b32
backend: Make pagination for List configurable
2017-09-18 12:01:54 +02:00
Alexander Neumann
01c486d486
Merge pull request #1250 from dvrkps/patch-1
...
travis: update go versions
2017-09-17 20:59:35 +02:00
Alexander Neumann
6342a08a16
Merge pull request #1248 from mungomat/fuse_typo
...
fuse: typo
2017-09-17 20:57:42 +02:00
Davor Kapsa
94c8ee11f8
travis: update go versions
2017-09-17 19:02:22 +02:00
Tobias Klein
9b38980ed9
fuse: typo
2017-09-17 17:39:28 +02:00
Alexander Neumann
649c536250
backend: Improve test for pagination in list
2017-09-17 11:36:45 +02:00
Alexander Neumann
dd49e2b12d
Azure: Fix List(), use pagination marker
2017-09-17 11:32:05 +02:00
Alexander Neumann
f61dab1774
backend: Add test for List()
2017-09-17 11:09:16 +02:00
Alexander Neumann
40edf00182
gs: implement pagination
2017-09-17 11:08:51 +02:00
Alexander Neumann
c35518a865
Azure/GS: Remove ReadDir()
2017-09-17 11:05:30 +02:00
Antoine Beaupré
7a0b4428e3
add explanation of restic automation
...
every time i look at restic, i block on this and figured it may be useful for others
2017-09-16 10:17:36 -04:00
Alexander Neumann
c784a15aaa
Merge pull request #1244 from restic/fix-swift-backend-tests
...
Ignore "not exist" errors for swift backend tests
2017-09-16 14:54:32 +02:00
Alexander Neumann
ce180de9b8
Merge pull request #1243 from restic/improve-error-reporting
...
Improve error reporting
2017-09-16 14:54:30 +02:00
Alexander Neumann
fca9a523e9
Merge pull request #1241 from restic/fix-timestamp-check
...
Use .Equal() instead of == for time.Time
2017-09-16 14:54:26 +02:00
Alexander Neumann
8a3889be11
Merge pull request #1240 from restic/config-autocomplete-dir
...
Correct bash completion file path
2017-09-16 14:54:20 +02:00
Alexander Neumann
2a1633621b
Ignore "not exist" errors for swift backend tests
2017-09-16 13:59:55 +02:00
Alexander Neumann
e2deeceb1b
Update manpage
2017-09-16 11:29:37 +02:00
Alexander Neumann
d4e994de7b
Improve error reporting
...
This will print the error (including a stack trace) if available before
exiting.
2017-09-16 10:55:13 +02:00
Alexander Neumann
a60e751217
Use .Equal() instead of == for time.Time
...
Closes #1238
2017-09-15 20:57:35 +02:00
Alexander Neumann
81c5d8a968
Correct bash completion file path
2017-09-15 20:45:16 +02:00
Alexander Neumann
5b1e4df177
Add version to CHANGELOG
2017-09-13 17:14:43 +02:00
Alexander Neumann
4d80744cbb
Add VERSION file for 0.7.2
2017-09-13 17:14:26 +02:00
Alexander Neumann
e243d4b7ee
Merge pull request #1236 from restic/update-deps
...
Update dependencies
2017-09-13 14:51:07 +02:00
Alexander Neumann
dce35fcb00
Merge pull request #1232 from prattmic/patch-1
...
doc: remove broken link
2017-09-13 14:18:46 +02:00
Alexander Neumann
e45a21b0b6
Merge pull request #1231 from restic/fix-local-datadir-create
...
local: Fix creating data dirs
2017-09-13 14:15:30 +02:00
Alexander Neumann
fda563d606
Update dependencies
2017-09-13 14:09:48 +02:00
Alexander Neumann
f3b49987f8
Add entry to CHANGELOG
2017-09-13 14:04:55 +02:00
Alexander Neumann
c8c01a5cae
Merge pull request #1223 from mrzv/snapshots-compact
...
Add --compact option to snapshots
2017-09-13 14:02:31 +02:00
Michael Pratt
f7ece90129
doc: remove broken link
...
The link to SFTP is broken, remove it.
I don't even bother to put a new link, since SFTP is literally the next section.
2017-09-12 21:02:34 -07:00
Alexander Neumann
0f25ef9498
Merge pull request #1230 from shayneholmes/update-short-help
...
Update style in short help commands
2017-09-11 22:25:31 +02:00
Alexander Neumann
5bf2228596
local: Fix creating data dirs
2017-09-11 21:48:25 +02:00
Alexander Neumann
227b01395f
local: Add test for open non-existing dir
2017-09-11 21:34:26 +02:00
Shayne Holmes
9f52fe1a10
Update manpages
...
This is a programmatic change; just ran `restic man-page`
2017-09-11 12:05:51 -07:00
Shayne Holmes
affc6c3390
Correct manpage command
2017-09-11 11:21:44 -07:00
Alexander Neumann
951a34dcbf
Add entry to CHANGELOG
2017-09-11 19:02:36 +02:00
Alexander Neumann
36eaa22ed0
Merge pull request #1205 from mungomat/backup_time
...
Backup time
2017-09-11 19:01:41 +02:00
Alexander Neumann
62df316356
Merge pull request #1194 from mungomat/bugfix_bucker_always
...
fix: bucker
2017-09-11 19:01:23 +02:00
Shayne Holmes
00797fdd85
Synchronize manual with help output
...
Two commands in the output weren't mentioned in the manual.
2017-09-11 09:33:19 -07:00
Shayne Holmes
9eb39cef05
Capitalize short help commands
...
Unify existing Cobra help command, and git-help's style.
2017-09-11 09:32:44 -07:00
Shayne Holmes
ee6150f67c
Change short help messages to imperative voice
...
Unify the output of `restic help`.
2017-09-11 09:26:13 -07:00
Alexander Neumann
9fa909ccd6
Update golden files
2017-09-11 17:52:22 +02:00
Alexander Neumann
b1af544b1d
Merge pull request #1224 from restic/improve-exclude-caches
...
Allow multiple exclude-if-present
2017-09-11 17:31:24 +02:00
Dmitriy Morozov
7d5b17ac72
Update man page for snapshots
2017-09-10 15:28:06 -07:00
Dmitriy Morozov
7a221f2473
Run changes through gofmt
2017-09-10 15:09:28 -07:00
Dmitriy Morozov
bdbe956c5c
Add --compact option to snapshots
...
With --compact, snapshots doesn't list directories and puts all tags on a single
line. This way each snapshot takes up exactly one line.
2017-09-10 13:06:43 -07:00
Alexander Neumann
8e5b1e6f2f
Add --group-by to manual
2017-09-10 21:00:51 +02:00
Alexander Neumann
257a454515
Add entry to CHANGELOG
2017-09-10 20:55:02 +02:00
Alexander Neumann
b6aeea425b
Merge pull request #1196 from mungomat/forget_groupByTagsOnly
...
forget: group-by-tags-only
2017-09-10 20:52:15 +02:00
Alexander Neumann
c8e05d1f2a
Add entry to CHANGELOG
2017-09-10 20:29:08 +02:00
Alexander Neumann
a8aa4eb06c
Rename parameter filename -> path
2017-09-10 20:28:21 +02:00
Alexander Neumann
c1a02cc081
Merge pull request #1228 from restic/fix-1204-3
...
Always use long name for keys
2017-09-10 19:39:23 +02:00
Alexander Neumann
e66adc42da
Always use long name for keys
...
Otherwise the code panics if a file with a short name is tried.
2017-09-10 15:35:10 +02:00
Alexander Neumann
89938bc21c
Update manual pages
2017-09-10 15:33:20 +02:00
Alexander Neumann
0b2947dedb
Add test for rejectByPattern
2017-09-10 15:31:58 +02:00
Alexander Neumann
47ddd34266
Improve test
2017-09-10 15:20:41 +02:00
Alexander Neumann
2fdca5d310
Improve debug message
2017-09-10 15:16:05 +02:00
Alexander Neumann
e5d4e33509
Improve error message if no targets specified
2017-09-10 15:14:11 +02:00
Alexander Neumann
e117f613af
Move device test into new RejectFunc
2017-09-10 15:13:40 +02:00
Alexander Neumann
0dfdf02885
Rework pattern excludes
2017-09-10 14:34:28 +02:00
Alexander Neumann
4a0129fc2b
Rename excludeByFile -> rejectIfPresent
2017-09-10 14:25:58 +02:00
Alexander Neumann
a9c705009c
Move reject functions to new file
2017-09-10 14:25:25 +02:00
Alexander Neumann
d937ad8cf6
Rename FilenameCheck to RejectFunc
...
We already have the opposite: pipe.SelectFunc(item string, fi
os.FileInfo) bool, so RejectFunc is a good name.
2017-09-10 14:21:51 +02:00
Alexander Neumann
1a08a8219f
Merge pull request #1227 from restic/fix-1204-2
...
Handle invalid key file
2017-09-10 14:12:51 +02:00
Tobias Klein
9924c311c9
added test cases
2017-09-10 12:23:28 +02:00
Alexander Neumann
e846e14965
Ignore files with invalid name in the repo
2017-09-10 11:00:07 +02:00
Alexander Neumann
36e70228f2
Handle invalid key file
2017-09-10 10:55:01 +02:00
Tobias Klein
a677f1139a
removed unnacessary line
2017-09-10 10:41:07 +02:00
Alexander Neumann
6f8eba9c28
Merge pull request #1222 from damiencourousse/manual-typo-fix
...
manual: typo fix
2017-09-09 21:40:00 +02:00
Alexander Neumann
c22c582546
Allow multiple exclude-if-present
2017-09-09 21:24:29 +02:00
Alexander Neumann
ea75509d6e
Print warning for non-existing items
2017-09-09 21:12:41 +02:00
Tobias Klein
ed30bd7b76
gofmt
2017-09-09 18:19:19 +02:00
Damien Couroussé
7090c5ceeb
manual: typo fix
2017-09-09 16:58:17 +02:00
Tobias Klein
bee09c1a0f
test
2017-09-09 16:33:51 +02:00
Tobias Klein
8f9ef4402b
error in case of unknown grouping option
2017-09-09 15:55:37 +02:00
Tobias Klein
f26c0cb70f
testcase updated
2017-09-09 15:33:12 +02:00
Tobias Klein
81d7ecba2b
manual updated
2017-09-09 13:26:35 +02:00
Tobias Klein
087c3fe1dc
tests updated
2017-09-09 13:26:35 +02:00
Tobias Klein
43ff971dfd
new sub-option for backup: time
...
New option to specify the timestamp for a backup
2017-09-09 13:26:35 +02:00
Alexander Neumann
5c75a98053
Merge pull request #1220 from restic/fix-1204
...
Fix panic when file name is too short
2017-09-09 11:49:23 +02:00
Alexander Neumann
7ce47402fb
Merge pull request #1170 from fawick/exclude_caches
...
Add option to exclude contents of cache directories
2017-09-09 10:56:12 +02:00
Alexander Neumann
1e48141648
Fix panic when file name is too short
...
Closes #1204
2017-09-09 10:50:32 +02:00
Fabian Wickborn
dbda892542
Add option to exclude directories with a tagfile
...
The option is named --exclude-if-present and accepts a parameter
filename[:content]. Directories are excluded and their contents is not
backed up if they contain a file with the specified name and,
optionally, that starts with the specified content. The tagfile itself
is never excluded.
There is also a shortcut --exclude-caches that works in the same way as
the likewise-named option of tar(1): Directories are recognized as cache
if they contain a file named "CACHEDIR.TAG.
Closes #317 .
2017-09-09 09:57:42 +02:00
Alexander Neumann
b46774be21
Merge pull request #1214 from ricardoseriani/fix-manual.rst-key-remove
...
Fix manual.rst to use key remove instead of key rm
2017-09-07 21:16:46 +02:00
Tobias Klein
1073bfba37
flexible grouping option for the forget-command
2017-09-06 20:14:18 +02:00
Alexander Neumann
5dfb4d1195
Merge pull request #1209 from restic/handle-colliding-names
...
Resolve name collisions
2017-09-05 22:33:32 +02:00
Alexander Neumann
0a2219c5f7
Travis: Reduce workload by removing Go tip
2017-09-05 22:08:23 +02:00
Alexander Neumann
ff3149831e
Merge pull request #1210 from prattmic/glob_test
...
filter: document recursive wildcards
2017-09-05 21:50:27 +02:00
Alexander Neumann
c935d0558c
Add entry to CHANGELOG
2017-09-05 21:48:13 +02:00
Alexander Neumann
83eb075e3a
Resolve name collisions
...
At the moment when two items to be saved have the same directory name,
restic only saves the first one to the repo. Let's say we have a
structure like this:
dir1
└── subdir
└── file
dir2
└── subdir
└── file
When restic is run on `dir1/subdir` and `dir2/subdir`, it will only save
the first `subdir`:
$ restic backup dir1/subdir dir2/subdir
[...]
$ restic ls -l latest
drwxr-xr-x 1000 100 0 2017-08-27 20:56:39 /subdir
-rw-r--r-- 1000 100 17 2017-08-27 20:56:39 /subdir/file
That's obviously a bad thing, caused by an early decision to strip the
full path to the files/dirs to save and only leave the last directory.
This commit partly resolves this by handling colliding names and
resolving the conflicts. Restic will now append a counter to the file
(`-123`) until the conflict is resolved. So in the example above, we'll
end up with the following structure:
$ restic ls -l latest
drwxr-xr-x 1000 100 0 2017-08-27 20:56:39 /subdir
-rw-r--r-- 1000 100 17 2017-08-27 20:56:39 /subdir/file
drwxr-xr-x 1000 100 0 2017-08-27 20:56:46 /subdir-1
-rw-r--r-- 1000 100 17 2017-08-27 20:56:46 /subdir-1/file
This partly addresses #549 and closes #1179 .
At first I thought that the obvious correction would be to archive the
full path. But it turns out that collisions may still occur: Suppose you
have a file named `foo` in the current directory, and the parent directory
also contains a file `foo`. Archiving these with restic also causes a
collision, since restic strips the `../` from the first file:
$ restic backup ../foo foo
This also happens with `tar`, which does not handle the collision and
will happily archive two files called `foo`.
So, the best way forward is to handle name collisions and archive the
whole path. The latter will be tackled in a separate PR.
2017-09-05 21:47:02 +02:00
Ricardo Seriani
204c2bf09c
Fix manual to use key remove instead of key rm
2017-09-05 16:22:42 -03:00
Alexander Neumann
2444522243
Add test for colliding names
2017-09-05 21:10:02 +02:00
Michael Pratt
92eb1cbffd
filter: document recursive wildcards
...
Match/ChildMatch accept a ** pattern which is not noted in the doc
string, nor do any of the docs or tests specify whether the match is
greedy (i.e., can 'foo/**/bar' match paths with additional intermediate
bar directories?).
Add a note to the doc string and add test cases for greedy matches.
2017-09-04 14:38:48 -07:00
Alexander Neumann
8c40ae5a03
Add entry to CHANGELOG
2017-09-04 21:58:33 +02:00
Alexander Neumann
fa2ee78a5c
Merge pull request #1044 from lloeki/982-improve-restore
...
Improve restore
2017-09-04 21:51:12 +02:00
Tobias Klein
e4a5cdc5bc
forget: group-by-tags-only
2017-09-03 17:11:25 +02:00
Tobias Klein
2d73a273af
saving a variable
2017-09-03 17:09:55 +02:00
Tobias Klein
761af08889
fix: bucker
...
bucker "always" does not return a unique id in case of exact same timestamps
2017-09-03 17:09:55 +02:00
Alexander Neumann
0ee1650f82
Merge pull request #1191 from prattmic/profile
...
debug: properly handle interrupted profiles
2017-09-03 09:49:46 +02:00
Alexander Neumann
0e647417f3
Add entry to CHANGELOG
2017-09-03 09:49:37 +02:00
Alexander Neumann
d1bf5a4882
Merge pull request #1203 from myfreeweb/master
...
Handle SIGINFO on all supported platforms
2017-09-03 09:44:24 +02:00
Alexander Neumann
b8414b240c
Add entry to CHANGELOG
2017-09-03 09:44:15 +02:00
Greg V
3fbdd12b04
Handle SIGINFO on all supported platforms
...
Not just darwin
2017-09-02 22:06:31 +03:00
Alexander Neumann
a3f6bf3e5a
Merge pull request #1202 from restic/fix-manpages
...
Fix manpages, update Go version
2017-09-02 10:11:43 +02:00
Alexander Neumann
3a5805db50
Update Go versions for CI
2017-09-02 09:29:02 +02:00
Alexander Neumann
de8c64e767
Use deterministic date for man pages
2017-09-02 09:27:11 +02:00
Alexander Neumann
73d6b15095
Merge pull request #1201 from gjmf/patch-1
...
Fixed word-o. ("Package", not "packet".)
2017-09-01 21:21:18 +02:00
Alexander Neumann
5d396b9302
Merge pull request #1200 from molivier/patch-1
...
Update manual.rst
2017-09-01 21:21:16 +02:00
Graham Freeman
61d2519111
Fixed word-o. ("Package", not "packet".)
...
Fixed a word-o. homebrew is a package manager, not a packet manager. :)
2017-08-31 13:34:53 -07:00
Matthieu OLIVIER
e61c94a846
Update manual.rst
...
`key remove` becomes `key rm`.
2017-08-31 18:26:04 +02:00
Alexander Neumann
7ed0f61f3f
Merge pull request #1189 from FiloSottile/patch-2
...
doc/design: fix keys.data MAC format description
2017-08-29 20:44:31 +02:00
Alexander Neumann
85055d1c68
Merge pull request #1187 from FiloSottile/patch-1
...
internal/crypto: small simplifications
2017-08-29 20:43:15 +02:00
Michael Pratt
e4c469c149
debug: properly handle interrupted profiles
...
By default (i.e., without profile.NoShutdownHook), profile.Start listens
for SIGINT and will stop the profile and call os.Exit(0).
restic already listens for SIGINT and runs its own cleanup handlers
before calling os.Exit(0).
As is, these handlers are racing when an interrupt occurs, and in my
experience, restic tends to win the race, resulting in an unusable
profile.
Eliminate the race and properly stop profiles on interrupt by disabling
package profile's signal handler and instead stop the profile in a
restic cleanup handler.
2017-08-28 22:03:26 -07:00
Filippo Valsorda
9940e8d9f1
internal/crypto: small simplifications
...
* append operates on len, not cap (not a bug since len is set to cap above, but let's avoid the confusion)
* no need to extend ciphertext again to cap after we made it big enough
* make consistent use of ciphertext[:ivSize] vs iv[:]
* make all input problems errors and impossible/catastrophic cases panics
2017-08-29 00:30:06 +02:00
Filippo Valsorda
3dccca1f27
doc/design: fix keys.data MAC format description
...
"not including the last 32 byte" was wrong, should have been 16 bytes. But the whole description is redundant anyway.
2017-08-29 00:22:11 +02:00
Alexander Neumann
22e96a37f8
Merge pull request #1184 from prattmic/docs
...
Doc cleanup and mention restore include/exclude
2017-08-28 21:14:48 +02:00
Alexander Neumann
48b1ab5aaf
Merge pull request #1182 from restic/fix-1167
...
local: do not create dirs below data/ for non-existing dir
2017-08-28 21:13:24 +02:00
Alexander Neumann
0230fa188f
Add entry to CHANGELOG
2017-08-28 21:13:14 +02:00
Alexander Neumann
4118ce876e
Merge pull request #1185 from prattmic/gcs_panic
...
gs: fix nil dereference
2017-08-28 21:11:30 +02:00
Michael Pratt
9537bc561d
gs: fix nil dereference
...
info can be nil if err != nil, resulting in a nil dereference while
logging:
$ # GCS config
$ ./restic init
debug enabled
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x935947]
goroutine 1 [running]:
github.com/restic/restic/internal/backend/gs.(*Backend).Save(0xc420012690, 0xe84e80, 0xc420010448, 0xb57149, 0x3, 0xc4203fc140, 0x40, 0xe7be40, 0xc4201d8f90, 0xa0, ...)
src/github.com/restic/restic/internal/backend/gs/gs.go:226 +0x6d7
github.com/restic/restic/internal/repository.AddKey(0xe84e80, 0xc420010448, 0xc4202f0360, 0xc42000a1b0, 0x4, 0x0, 0xa55b60, 0xc4203043e0, 0xa55420)
src/github.com/restic/restic/internal/repository/key.go:235 +0x4a1
github.com/restic/restic/internal/repository.createMasterKey(0xc4202f0360, 0xc42000a1b0, 0x4, 0xa55420, 0xc420304370, 0x6a6070)
src/github.com/restic/restic/internal/repository/key.go:62 +0x60
github.com/restic/restic/internal/repository.(*Repository).init(0xc4202f0360, 0xe84e80, 0xc420010448, 0xc42000a1b0, 0x4, 0x1, 0xc42030a440, 0x40, 0x32a4573d3d9eb5, 0x0, ...)
src/github.com/restic/restic/internal/repository/repository.go:403 +0x5d
github.com/restic/restic/internal/repository.(*Repository).Init(0xc4202f0360, 0xe84e80, 0xc420010448, 0xc42000a1b0, 0x4, 0xe84e40, 0xc42004ad80)
src/github.com/restic/restic/internal/repository/repository.go:397 +0x12c
main.runInit(0xc420018072, 0x16, 0x0, 0x0, 0x0, 0xe84e40, 0xc42004ad80, 0xc42000a1b0, 0x4, 0xe7dac0, ...)
src/github.com/restic/restic/cmd/restic/cmd_init.go:47 +0x2a4
main.glob..func9(0xeb5000, 0xedad70, 0x0, 0x0, 0x0, 0x0)
src/github.com/restic/restic/cmd/restic/cmd_init.go:20 +0x8e
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).execute(0xeb5000, 0xedad70, 0x0, 0x0, 0xeb5000, 0xedad70)
src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:649 +0x457
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xeb3e00, 0xc420011650, 0xa55b60, 0xc420011660)
src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:728 +0x339
github.com/restic/restic/vendor/github.com/spf13/cobra.(*Command).Execute(0xeb3e00, 0x25, 0xc4201a7eb8)
src/github.com/restic/restic/vendor/github.com/spf13/cobra/command.go:687 +0x2b
main.main()
src/github.com/restic/restic/cmd/restic/main.go:72 +0x268
(The error was likely because I had just enabled the GCS API. Subsequent
runs were fine.)
2017-08-27 21:36:04 -07:00
Michael Pratt
ae43c47ca8
doc: add mention of restore --exclude/--include
...
There is a lot more detail that could be added here, but it is worth
getting things off the ground with at least a mention that it is
possible to restore individual files.
Updates #396
2017-08-27 18:36:00 -07:00
Michael Pratt
2fa4060991
doc: fix inconsistencies
...
* Replace references to ~/shared/work/web which should be ~/web.
* Replace references to ~/tmp which should be /tmp.
* Restore /home/art to /tmp/restore-art instead of /tmp/restore-work,
which is clearly a copy/paste from the command above.
2017-08-27 18:35:01 -07:00
Alexander Neumann
f9a934759f
sftp: Improve error handling for non-existing dir
2017-08-27 20:53:04 +02:00
Alexander Neumann
3686b1ffe5
local: Create directories below data/ if it exists
2017-08-27 20:52:58 +02:00
Alexander Neumann
ea017a49c3
local: Add test for #1167
...
It was discovered that restic creates directories when a non-existing
directory is specified as a local repository.
2017-08-27 20:38:46 +02:00
Alexander Neumann
3559f9c776
Merge pull request #1174 from pwaring/patch-1
...
Update minimum Go version
2017-08-25 21:26:13 +02:00
Paul Waring
637f57ca71
Update minimum Go version
...
Minimum version is now 1.8 according to build.go (from latest master, cloned a few minutes ago):
```
paul@voga:~/third-dev/restic$ go run build.go
Go version go1.7.4 detected, restic requires at least Go 1.8
exit status 1
```
2017-08-24 19:51:11 +01:00
Alexander Neumann
4e60156b45
Add entry to CHANGELOG
2017-08-18 19:50:32 +02:00
Alexander Neumann
af9946b098
Merge pull request #1164 from ricardoseriani/fix-key-remove-command
...
Change key rm command to key remove
2017-08-18 19:49:50 +02:00
Ricardo Seriani
b7d4b0f821
Update man pages
2017-08-17 16:44:28 -03:00
Ricardo Seriani
62ed776a8c
Change key rm command to key remove
...
Change key rm command to key remove, to follow manual and other commands
2017-08-17 11:03:26 -03:00
Loic Nageleisen
f880ff21aa
Fixing restore with excluded
...
An exclude filter is basically a 'wildcard but foo', so even if a
childMayMatch, other children of a dir may not, therefore childMayMatch
does not matter, but we should not go down unless the dir is selected
for restore.
2017-08-16 15:25:02 +02:00
Loic Nageleisen
4a36993c19
Smarter filter when children won't match
...
This improves restore performance by several orders of magniture by not
going through the whole tree recursively when we can anticipate that no
match will ever occur.
2017-08-16 15:25:02 +02:00
Alexander Neumann
d87b2f189d
Merge pull request #1157 from ceh/defer-file-close-after-err-check
...
internal: check error before deferring file Close()
2017-08-13 19:50:05 +02:00
Alexander Neumann
f9a097a8c0
Merge pull request #1158 from ceh/fix-contributing-typo
...
Fix contribution typo
2017-08-13 19:49:03 +02:00
Alexander Neumann
d43358b6dd
Correct URL to forum
2017-08-13 19:47:54 +02:00
Alexander Neumann
8058f196e1
Merge pull request #1156 from dimejo/patch-1
...
Readme: Correct link formatting
2017-08-13 19:47:31 +02:00
Emil Hessman
e13e6f34d2
Fix contribution typo
2017-08-13 19:35:53 +02:00
Emil Hessman
c2ff7150aa
internal: check error before deferring file Close()
...
If there is an error, file will be `nil`. We should check the returned error before deferring file `Close()`.
2017-08-13 19:28:13 +02:00
dimejo
a899621930
Readme: Correct link formatting
2017-08-13 15:07:40 +02:00
Alexander Neumann
a0966e1d1d
Update README
2017-08-11 21:38:18 +02:00
Alexander Neumann
e2464382ed
Update issue template
2017-08-11 21:36:10 +02:00
Alexander Neumann
095bc79dc3
Correct README #2
2017-08-09 21:58:20 +02:00
Alexander Neumann
1fd3c2488e
Correct README
2017-08-09 21:57:55 +02:00
Alexander Neumann
2ee8485886
Update README
2017-08-09 21:56:41 +02:00
Alexander Neumann
b67c178672
Merge pull request #1149 from restic/azure-support
...
Add Azure blob storage as backend
2017-08-09 21:30:35 +02:00
Alexander Neumann
7ac4f0a525
Merge pull request #1134 from restic/gcs-support
...
Add backend for Google Cloud Storage (GCS)
2017-08-09 21:28:36 +02:00
Alexander Neumann
c4613c51d1
Add note about Go 1.8
2017-08-09 20:33:30 +02:00
Alexander Neumann
77bf17076b
Add entry to CHANGELOG
2017-08-09 20:23:29 +02:00
Alexander Neumann
8dd6beba15
gs: Add section to the manual
2017-08-09 20:22:07 +02:00
Alexander Neumann
a345386967
Add a section to the CHANGELOG
2017-08-09 20:15:08 +02:00
Alexander Neumann
bdd43bd430
Add a section to the manual
2017-08-09 20:13:34 +02:00
Alexander Neumann
1716501598
CI: Make sure the GCS backend tests run on Travis
2017-08-06 21:47:56 +02:00
Alexander Neumann
d9a5b9178e
gs: Rework path initialization
2017-08-06 21:47:56 +02:00
Alexander Neumann
8ca6a9a240
Vendor dependencies for GCS
2017-08-06 21:47:56 +02:00
Dipta Das
ba75a3884c
Add Google Cloud Storage as backend
...
Environment variables:
GOOGLE_PROJECT_ID=gcp-project-id
GOOGLE_APPLICATION_CREDENTIALS=path-to-json-file
Environment variables for test:
RESTIC_TEST_GS_PROJECT_ID=gcp-project-id
RESTIC_TEST_GS_APPLICATION_CREDENTIALS=path-to-json-file
RESTIC_TEST_GS_REPOSITORY=gs:us-central1/test-bucket
Init repository:
$ restic -r gs🪣 /[prefix] init
2017-08-06 21:47:55 +02:00
Alexander Neumann
d91d89eef6
azure: Create container if it does not exist
2017-08-06 21:47:04 +02:00
Alexander Neumann
a726c91116
azure: Rework path initialization
2017-08-06 21:47:04 +02:00
Alexander Neumann
d00fe95f10
Upgrade min Go version to 1.8
2017-08-06 21:47:04 +02:00
Alexander Neumann
072b7a014e
azure: User internal errors package
2017-08-06 21:47:04 +02:00
Alexander Neumann
618ce115d7
Azure: Use default HTTP transport
2017-08-06 21:47:04 +02:00
Alexander Neumann
d973aa82fe
Vendor dependencies for azure backend
2017-08-06 21:47:04 +02:00
Dipta Das
3a85b6b7c6
Add Azure Blob Storage as backend
...
Environment variables:
AZURE_ACCOUNT_NAME=storage-account-name
AZURE_ACCOUNT_KEY=storage-account-key
Environment variables for test:
RESTIC_TEST_AZURE_ACCOUNT_NAME=storage-account-name
RESTIC_TEST_AZURE_ACCOUNT_KEY=storage-account-key
RESTIC_TEST_AZURE_REPOSITORY=azure:restic-test-container
Init repository:
$ restic -r azure:container-name:/prefix/dir init
2017-08-06 21:47:04 +02:00
Alexander Neumann
2c22ff175c
Merge pull request #1150 from restic/removed-date-from-manpage
...
Fix manpage generation
2017-08-06 21:46:44 +02:00
Alexander Neumann
6bc43a4198
manpage: Remove auto gen tag from man page
2017-08-06 21:31:01 +02:00
Alexander Neumann
e348b3deeb
manpage: Do not panic when no command is given
2017-08-06 21:01:49 +02:00
Alexander Neumann
6724b9a583
Merge pull request #1148 from restic/update-simple-scrypt
...
Lock simple-scrypt library to master branch
2017-08-05 20:04:33 +02:00
Alexander Neumann
41c35b2218
Lock simple-scrypt library to master branch
...
The master branch includes a fix for i386, otherwise the calibration
panics. See https://github.com/restic/restic/issues/676 for details.
2017-08-05 19:24:56 +02:00
Alexander Neumann
4477d76f03
Merge pull request #1147 from restic/add-manpage
...
Add new command 'manpage'
2017-08-05 12:39:42 +02:00
Alexander Neumann
14f5f6235a
Add entry to CHANGELOG
2017-08-05 12:05:53 +02:00
Alexander Neumann
739350fd8e
backup: Do not print hostname in help text
...
This is necessary so that the manpage generation is deterministic and we
can test if the man pages are up to date when the CI tests run.
2017-08-05 12:05:53 +02:00
Alexander Neumann
14ed97102b
Add instructions for developers
2017-08-05 12:05:53 +02:00
Alexander Neumann
db389058fa
Add generated manual pages
2017-08-05 12:05:53 +02:00
Alexander Neumann
b557d04007
CI: Fix test for forbidden packages
2017-08-05 12:05:53 +02:00
Alexander Neumann
52c5da997b
Add CI test for manpages
2017-08-05 12:05:53 +02:00
Alexander Neumann
57d198f99a
Vendor dependencies for manpage generation
2017-08-05 11:08:49 +02:00
Alexander Neumann
a3ab17b470
Add 'manpage' command to generate manual pages
2017-08-05 10:57:01 +02:00
Alexander Neumann
9bf3141893
Add entry to CHANGELOG
2017-08-01 22:02:08 +02:00
Alexander Neumann
d35eb6a0c3
Merge pull request #1144 from wjkohnen/close-exclude-files
...
Close exclude files and check errors
2017-08-01 22:01:43 +02:00
Johannes Kohnen
37aad2e3aa
Close exclude files and check errors
2017-08-01 17:34:27 +00:00
Alexander Neumann
efc5d0699a
Merge pull request #1139 from donat-b/pwfile-doesnt-exist
...
Error message in case PasswordFile is missing
2017-07-27 15:41:50 +02:00
donat
893bc9f777
Error message in case PasswordFile is missing
2017-07-27 14:23:08 +03:00
Alexander Neumann
61b8729ef9
Merge pull request #1138 from stapelberg/patch-1
...
fix typo: explicitely → explicitly
2017-07-27 09:01:47 +02:00
Michael Stapelberg
b89d3cc4d0
fix typo: explicitely → explicitly
2017-07-27 08:24:53 +02:00
Alexander Neumann
e8cc11ea34
Add entry to CHANGELOG
2017-07-26 22:29:39 +02:00
Alexander Neumann
2e804511ca
Merge pull request #1133 from middelink/fix-1132
...
Force restic to ask the password when adding a key.
2017-07-26 22:11:37 +02:00
Alexander Neumann
b6790c491b
Merge pull request #1131 from restic/test-cleanups
...
Rework withTestEnvironment
2017-07-26 22:06:06 +02:00
Pauline Middelink
c95e2b009e
Simpify cmd_backup and cmd_init now we have the password in gopts.
2017-07-24 23:32:55 +02:00
Pauline Middelink
d5615a67c8
Refactor password resolving.
...
Instead of determining the password lazily during ReadPassword(), do so now in
cobra.PersistentPreRunE() so we can store the result in the globalOptions and
reuse/override when applicable without having to worry about the environment
or flag options interfering.
2017-07-24 23:05:37 +02:00
Pauline Middelink
d9b9bbd4a8
Force restic to ask the password when adding a key.
...
As `restic key add` uses the same `ReadPasswordTwice()` as the
rest of restic, it is sensitive to the environment variable
RESTIC_PASSWORD or --password-file= override.
When asking for the new key, temporary remove these 2 overrides, forcing
the password to be asked.
2017-07-24 22:00:44 +02:00
Alexander Neumann
d780b1eede
Rework withTestEnvironment
...
Switch from a function passed as a parameter to a cleanup function,
which is also executed when the test function panics, so no temporary
directories are left behind.
2017-07-24 21:32:34 +02:00
Alexander Neumann
608adf15a3
Merge pull request #1130 from middelink/fix-fuse-test
...
Fuse testing leaves test mountpoint around.
2017-07-24 21:26:05 +02:00
Pauline Middelink
1717391f6c
Fuse testing leaves test mountpoint around. Move it under the testing tree which is removed after each test.
2017-07-24 20:33:39 +02:00
Alexander Neumann
2e6e9ff6f8
Merge pull request #1129 from restic/move-restic-pkg
...
Move restic package to internal/restic
2017-07-24 19:19:28 +02:00
Alexander Neumann
23c903074c
Move restic package to internal/restic
2017-07-24 17:43:32 +02:00
Alexander Neumann
94030a12cf
Add entry to CHANGELOG
2017-07-23 16:54:56 +02:00
Alexander Neumann
f63d7de9da
Merge pull request #1126 from restic/switch-to-default-go-git-layout
...
Use idiomatic default Go git repo layout
2017-07-23 16:54:53 +02:00
Alexander Neumann
13ee6792df
Add remark about GOPATH on forked repos
2017-07-23 16:36:13 +02:00
Alexander Neumann
6302444f34
Remove linebreak from Errorf() format string
2017-07-23 15:51:44 +02:00
Alexander Neumann
61c5e4b54a
Fix glyphcheck
2017-07-23 15:51:39 +02:00
Alexander Neumann
d6118871be
Update other files
2017-07-23 14:40:05 +02:00
Alexander Neumann
94b27e8933
Fix paths for tests
2017-07-23 14:39:57 +02:00
Alexander Neumann
05500dc5f8
Update documentation
2017-07-23 14:32:31 +02:00
Alexander Neumann
c5a72971fe
Remove Vagrantfile
2017-07-23 14:30:33 +02:00
Alexander Neumann
5bc6486e3b
Update docs
2017-07-23 14:25:39 +02:00
Alexander Neumann
59e18bce0a
Fix build.go
2017-07-23 14:25:39 +02:00
Alexander Neumann
898c5b6df5
Fix integration tests
2017-07-23 14:25:39 +02:00
Alexander Neumann
9cd422791a
Update build.go
2017-07-23 14:25:38 +02:00
Alexander Neumann
91edebf1fe
Vendor dependencies with dep
2017-07-23 14:25:38 +02:00
Alexander Neumann
df8a5792f1
Remove Dockerfile
2017-07-23 14:25:38 +02:00
Alexander Neumann
cda7b417cd
Remove envrc
2017-07-23 14:25:38 +02:00
Alexander Neumann
d2ac35af26
Remove vendor
2017-07-23 14:25:37 +02:00
Alexander Neumann
6caeff2408
Run goimports
2017-07-23 14:21:03 +02:00
Alexander Neumann
83d1a46526
Moves files
2017-07-23 14:19:13 +02:00
Alexander Neumann
d1bd160b0a
Merge pull request #1061 from bclermont/docker-image
...
add docker image
2017-07-22 11:52:22 +02:00
Alexander Neumann
bc88a8bb03
Add entry to CHANGELOG
2017-07-22 11:52:16 +02:00
Alexander Neumann
04cfb984ae
Add VERSION file for 0.7.1
2017-07-22 11:04:32 +02:00
Alexander Neumann
02a245941a
Adapt CHANGELOG for 0.7.1
2017-07-22 11:03:44 +02:00
Alexander Neumann
7fb1352aa1
Merge pull request #1124 from restic/use-minio-300
...
Set minio-go to v3.0.0
2017-07-22 11:01:57 +02:00
Alexander Neumann
4c555bad2e
Set minio-go to v3.0.0
2017-07-22 10:19:52 +02:00
Alexander Neumann
75c789bab4
Merge pull request #1122 from restic/swift-remove-range-test
...
swift: Remove check for byte range
2017-07-21 23:05:05 +02:00
Alexander Neumann
626d020e62
swift: Remove check for byte range
...
Closes #1084
Closes #1094
2017-07-21 20:45:25 +02:00
Alexander Neumann
3830117735
Merge pull request #1121 from restic/fix-swift-test
...
swift: Increase backend test delay for removed file
2017-07-21 20:37:10 +02:00
Alexander Neumann
042cee8e36
Merge pull request #1117 from donat-b/password-file-env
...
Set default value for password-file flag from env
2017-07-21 19:47:49 +02:00
Alexander Neumann
03cc5b47e9
appveyor: Update Go version to 1.8.3
2017-07-21 19:42:34 +02:00
Alexander Neumann
46fa45942e
swift: Increase backend test delay for removed file
2017-07-21 19:42:34 +02:00
Alexander Neumann
0cb4104aa7
Fix Go report card URLs (thanks @tyll)
2017-07-20 22:31:41 +02:00
donat
f2bbc5fbc4
Set default value for password-file flag from env
...
Allows defining password file path as RESTIC_PASSWORD_FILE=/foo
2017-07-20 10:47:02 +03:00
Alexander Neumann
16340ce811
Merge pull request #1090 from middelink/fix-1081
...
Update HasTags() and HasPaths() to follow #1081 feature request
2017-07-19 17:11:18 +02:00
Alexander Neumann
2cf8153f4a
Add entry to CHANGELOG
2017-07-19 17:09:02 +02:00
Alexander Neumann
2f00287e45
Merge pull request #1112 from restic/fix-chmod-not-supported
...
Ignore error for Chmod() on FS that don't support it
2017-07-19 17:05:18 +02:00
Alexander Neumann
0de17f64e9
Add entry to CHANGELOG
2017-07-19 17:02:53 +02:00
Alexander Neumann
c30838878f
Merge pull request #1115 from restic/fix-prune-index
...
prune: Fix newly created index
2017-07-19 17:01:11 +02:00
Alexander Neumann
bd31281f1e
prune: Fix newly created index
...
This fixes a bug introduced in c79fb6fcdd
where the index file after a prune contains pack files that do not exist
any more. Restic will detect this during backup and abort with an error
message until the user runs `prune` or `rebuild-index` again.
2017-07-18 23:10:30 +02:00
Alexander Neumann
7fc54ed98e
Improve test case for `prune`
2017-07-18 23:07:29 +02:00
Alexander Neumann
0abdcedcab
Ignore error for Chmod() on FS that don't support it
...
See #1079
2017-07-18 21:47:30 +02:00
Alexander Neumann
6c05353086
Add entry to CHANGELOG
2017-07-17 22:02:06 +02:00
Alexander Neumann
e7575bf380
Merge pull request #1108 from restic/update-deps
...
Update vendored deps
2017-07-17 22:01:52 +02:00
Alexander Neumann
89ace85903
s3: Use streaming API and remove workarounds
2017-07-17 20:43:53 +02:00
Alexander Neumann
68a91d66b7
s3: Use new API for CopyObject
2017-07-17 20:43:45 +02:00
Alexander Neumann
724b5bf4fe
Update minio-go
2017-07-17 20:19:04 +02:00
Alexander Neumann
d6da9211bc
Update vendored deps (except minio-go)
2017-07-17 20:00:44 +02:00
Alexander Neumann
f45abac27f
Merge pull request #1107 from bclermont/fix-s3-panic
...
Fix S3 panic on Invalid configuration
2017-07-17 18:50:46 +02:00
Bruno Clermont
00b9a1d87d
evaluate open error
2017-07-17 11:33:19 +03:00
Alexander Neumann
20b835b5a4
Improve help text
2017-07-16 21:41:13 +02:00
Alexander Neumann
7bb1a474df
Add entry to CHANGELOG
2017-07-16 21:41:13 +02:00
Alexander Neumann
750ee35dbf
Add more examples to the manual
2017-07-16 21:40:53 +02:00
Alexander Neumann
fda5e1f543
Adress code review comments
2017-07-16 21:40:53 +02:00
Alexander Neumann
78d090aea5
Implement TagList and TagLists as pflag.Value
2017-07-16 21:40:53 +02:00
Alexander Neumann
7362569cf5
Use TagLists for all commands
2017-07-16 21:40:53 +02:00
Alexander Neumann
f5b1c7e5f1
Add TagList
2017-07-16 21:40:53 +02:00
Pauline Middelink
c554cdac4c
Update HasTags() and HasPaths() to follow #1081 idea
...
Replace all but 3 occurences of StringSliceVar to StringArrayVar. This will
prevent the flag parser to interpretate the given values as CSV string.
Both --tag, --keep-tag and --path can be given multiple times, the command will
match snapshots matching ANY of the tags/paths. Only when a value is given which
contains a comma separated list of tags/paths, ALL elements need to match.
2017-07-16 21:40:53 +02:00
Alexander Neumann
41b624ea1b
Merge pull request #1105 from restic/improve-sftp-open
...
sftp: Improve check for data subdirs
2017-07-16 15:41:13 +02:00
Alexander Neumann
7cdcaadcf5
Add entry to CHANGELOG
2017-07-16 15:11:26 +02:00
Alexander Neumann
4ad33d3c3b
sftp: Improve check for data subdirs
2017-07-16 15:10:06 +02:00
Alexander Neumann
2778ac21de
Merge pull request #1103 from tobya/docsupdate
...
Update links to design.md to design.rst
2017-07-16 10:39:53 +02:00
Toby Allen
cb3cd57926
Update links to design.md to design.rst
2017-07-15 19:35:45 +01:00
Alexander Neumann
ba6815d413
Merge pull request #1100 from fawick/master
...
Allow absolute target path in build.go
2017-07-15 10:11:45 +02:00
Fabian Wickborn
52004cdde8
Allow absolute target path in build.go
...
Fixes #1099 .
2017-07-14 11:54:46 +02:00
Alexander Neumann
1d2045cb61
Test error for os.PathError
...
See https://github.com/restic/restic/issues/1079#issuecomment-315177469
for details.
2017-07-13 21:29:29 +02:00
Alexander Neumann
357e2e404a
Merge pull request #1080 from restic/fix-1079
...
local: Ignore ENOTSUP error for chmod
2017-07-13 20:12:47 +02:00
Alexander Neumann
38e5640cda
Add CHANGELOG entry
2017-07-09 21:43:05 +02:00
Alexander Neumann
c4c731bd9a
Merge pull request #1082 from Habbie/siginfo
...
support SIGINFO on Darwin
2017-07-09 21:41:29 +02:00
Alexander Neumann
04d27acd60
Add entry to CHANGELOG
2017-07-05 20:54:37 +02:00
Alexander Neumann
80f0303b21
Merge pull request #1086 from kamsz/iam
...
Add support for IAM instance profile
2017-07-05 20:53:31 +02:00
Kamil Szczygieł
d651d9b427
more verbose debug
2017-07-05 19:21:57 +02:00
Kamil Szczygieł
3b2648bd5e
iam instance profile
2017-07-05 16:19:25 +02:00
Peter van Dijk
73cc11f000
support SIGINFO on Darwin
2017-07-03 20:39:42 +02:00
Alexander Neumann
637de0149c
Add entry to CHANGELOG
2017-07-03 19:49:18 +02:00
Alexander Neumann
855575e5a7
Merge pull request #1077 from restic/create-subdirs
...
local/sftp: Auto-create subdirs of data/ on init/open
2017-07-03 19:47:58 +02:00
Alexander Neumann
ed2999a163
Merge pull request #1075 from restic/migrate-s3-continue
...
s3: Improve migration to new layout
2017-07-03 19:47:22 +02:00
Alexander Neumann
a18c16e19e
local: Ignore ENOTSUP error for chmod
...
Closes : #1079
2017-07-03 19:45:56 +02:00
Alexander Neumann
9032ab2eec
local/sftp: Create dirs on open()
2017-07-02 19:35:45 +02:00
Alexander Neumann
03f66b8d74
Create subdirs below data/
2017-07-02 19:35:45 +02:00
Alexander Neumann
8c30ae7c65
Add entry to CHANGELOG
2017-07-02 11:21:05 +02:00
Alexander Neumann
453c9c9199
s3 migrate layout: Retry on errors
2017-07-02 11:15:20 +02:00
Alexander Neumann
993e370f92
s3 migrate layout: Ignore already renamed files
2017-07-02 10:47:50 +02:00
Alexander Neumann
2bcd3a3acc
s3 migrate layout: Rename key files last
2017-07-02 10:47:20 +02:00
Alexander Neumann
c54c632ca1
s3 migrate layout: Force old layout for rename
2017-07-02 10:47:03 +02:00
Alexander Neumann
28a4a35625
Allow migrate to run althoug check failed
2017-07-02 10:29:41 +02:00
Alexander Neumann
e7577d7bb4
Add stub to CHANGELOG
2017-07-01 15:11:36 +02:00
Alexander Neumann
27ea0623d7
Add VERSION file for 0.7.0
2017-07-01 14:12:07 +02:00
Alexander Neumann
390e2bbddc
Merge pull request #1070 from restic/warn-unsupported-repo-type
...
Return an error for invalid backend schemes
2017-06-30 22:15:17 +02:00
Alexander Neumann
b50fc08f39
Add entry to CHANGELOG
2017-06-30 22:15:00 +02:00
Alexander Neumann
b2ce7e8d84
Return an error for invalid backend schemes
...
Closes #1021
2017-06-30 21:28:39 +02:00
Alexander Neumann
2b1c6d3cf8
Merge pull request #1066 from restic/update-minio-go
...
Update minio-go
2017-06-30 20:40:43 +02:00
Alexander Neumann
c658305a1b
Correct path for rest-server
2017-06-27 21:19:48 +02:00
Alexander Neumann
63235d8f94
Update minio-go
2017-06-26 22:06:57 +02:00
Bruno Clermont
d702227af0
install fuse and ca-certificates
2017-06-23 10:38:19 +02:00
Bruno Clermont
b7251dbea5
add docker image
2017-06-23 10:28:18 +02:00
Alexander Neumann
144b7f3386
doc: Correct path in manual
2017-06-22 19:54:55 +02:00
Alexander Neumann
9583dc820f
Merge pull request #1051 from restic/refactor-crypto
...
crypto: Make Encrypt/Decrypt a method of *Key
2017-06-21 19:26:11 +02:00
Alexander Neumann
a03076f2d8
Merge pull request #1056 from restic/fix-1053
...
prune: Delete invalid/incomplete pack files
2017-06-21 19:25:55 +02:00
Alexander Neumann
d76fa22b4b
prune: Delete invalid/incomplete pack files
...
Closes #1053
2017-06-20 22:53:49 +02:00
Alexander Neumann
f960831f10
crypto: Make Encrypt/Decrypt a method of *Key
2017-06-20 22:14:51 +02:00
Alexander Neumann
b0fb95dfc9
backend tests: Use delayedRemove()
2017-06-19 20:02:49 +02:00
Alexander Neumann
bca9566849
Merge pull request #1050 from restic/extend-fuse-mount
...
fuse: Add more directories
2017-06-19 19:52:45 +02:00
Alexander Neumann
8760de42fe
Merge pull request #1046 from restic/s3-split-open
...
s3: Split Create() from Open()
2017-06-19 19:52:40 +02:00
Alexander Neumann
2c02efd1fe
fuse: Reduce code duplication, add MetaDir
2017-06-18 21:32:07 +02:00
Alexander Neumann
4b4a63ed44
fuse: Add tags dir
2017-06-18 21:32:07 +02:00
Alexander Neumann
64f434eca4
fuse: Add hosts dir
2017-06-18 21:32:07 +02:00
Alexander Neumann
f4e85a53e7
fuse: Add '.' and '..' entries to all directories
2017-06-18 21:32:07 +02:00
Alexander Neumann
f8176a74ec
fuse: Rename DirSnapshots -> SnapshotsDir
2017-06-18 21:32:07 +02:00
Alexander Neumann
e60a96a71a
swift: Increase delete timeout to 20s
2017-06-18 21:31:48 +02:00
Alexander Neumann
216e2607ca
Add entry to CHANGELOG
2017-06-18 21:18:11 +02:00
Alexander Neumann
53f8026018
Merge pull request #1048 from restic/cleanup-fuse-mount
...
Cleanup/fix fuse mount
2017-06-18 18:41:02 +02:00
Alexander Neumann
de92ce7a88
Merge pull request #1049 from restic/fix-backend-tests-delayed-remove
...
backend tests: Add configurable delay for delayed remove
2017-06-18 18:31:38 +02:00
Alexander Neumann
eb8041b943
backend tests: Add configurable delay for delayed remove
2017-06-18 17:36:57 +02:00
Alexander Neumann
9c6e9bcf33
fuse: Add build tags for unsupported OS
2017-06-18 17:02:07 +02:00
Alexander Neumann
154816ffd0
fuse: Fix file test
2017-06-18 16:29:00 +02:00
Alexander Neumann
c86e425df6
fuse: Fix file inode
2017-06-18 16:28:55 +02:00
Alexander Neumann
3883c7a190
fuse: Fix blob length cache
2017-06-18 16:28:39 +02:00
Alexander Neumann
a66760d86d
fuse: Fix inode handling
2017-06-18 15:11:32 +02:00
Alexander Neumann
52752659c1
fuse: Rewrite fuse implementation
2017-06-18 14:59:44 +02:00
Alexander Neumann
f676c0c41b
index: Add Each() to MasterIndex
2017-06-18 14:52:14 +02:00
Alexander Neumann
f31e993f09
fuse: Reenable integration tests
2017-06-18 14:23:35 +02:00
Alexander Neumann
56f610e548
fuse: Remove struct SnapshotWithId
2017-06-18 14:11:33 +02:00
Alexander Neumann
052a6a0acc
Move snapshot filter function to restic package
2017-06-18 13:18:12 +02:00
Alexander Neumann
77037e33c9
Move snapshot finding functions to new file
2017-06-18 13:06:52 +02:00
Alexander Neumann
5a34799554
Move Snapshots struct and policy to other files
2017-06-18 13:05:47 +02:00
Alexander Neumann
47282abfa4
fuse: Use Mutex instead of RWMutex
2017-06-17 23:00:38 +02:00
Alexander Neumann
c9cc724b31
s3: Split Create() from Open()
2017-06-17 22:15:58 +02:00
Alexander Neumann
0d3674245b
Merge pull request #1043 from restic/fix-gcs
...
s3: Fix GCS
2017-06-17 10:35:10 +02:00
Alexander Neumann
82b21cdf4a
Merge pull request #1027 from restic/s3-set-retry
...
s3: Allow setting the number of retries for minio-go
2017-06-17 10:34:36 +02:00
Alexander Neumann
c4592f577a
Merge pull request #1036 from restic/prune-remove-invalid-files
...
prune: Remove invalid files
2017-06-16 22:52:44 +02:00
Alexander Neumann
3cd851e578
Update github.com/minio/minio-go
2017-06-16 22:29:40 +02:00
Alexander Neumann
e074833a7d
Merge pull request #1045 from restic/prune-fix-progress
...
prune: Fix progress information
2017-06-16 20:21:55 +02:00
Alexander Neumann
c5f1a83cb4
prune: Fix progress information
2017-06-16 19:03:26 +02:00
Alexander Neumann
1baaa778ee
Add entry to CHANGELOG
2017-06-16 12:27:44 +02:00
Alexander Neumann
6a948d5afd
s3: Fix backend for Google Cloud Storage
2017-06-16 11:25:06 +02:00
Alexander Neumann
ea66ae0811
s3: Fix IsNotExist()
2017-06-16 10:54:46 +02:00
Alexander Neumann
bf8a155fb1
Update github.com/minio/minio-go
2017-06-16 10:53:38 +02:00
Alexander Neumann
4ae59bef96
prune: Remove invalid files
...
Closes #1029
2017-06-15 20:56:22 +02:00
Alexander Neumann
eadf5dcb2d
Merge pull request #1038 from restic/s3-prevent-close
...
Improve GCS support
2017-06-15 20:54:52 +02:00
Alexander Neumann
91a24e8229
Merge pull request #1035 from restic/fix-1032
...
prune: Remove files as the last step
2017-06-15 20:22:42 +02:00
Alexander Neumann
e3c979a7a4
Merge pull request #1034 from restic/fix-1030
...
prune: Fix status string for narrow terminals
2017-06-15 20:22:33 +02:00
Alexander Neumann
05365706c0
backend/tests: Correct error message and delayed remove
2017-06-15 20:05:35 +02:00
Alexander Neumann
bbca31b661
test/s3: Retry connection to Minio server
2017-06-15 19:51:55 +02:00
Alexander Neumann
eb7fc12e01
backend tests: Delay listing for swift backend
2017-06-15 19:41:07 +02:00
Alexander Neumann
98ae7b1210
s3: Save config in backend
2017-06-15 16:41:09 +02:00
Alexander Neumann
51877cecf7
s3: Prevent closing of the reader for GCS
2017-06-15 16:39:42 +02:00
Alexander Neumann
9053b2000b
s3: Delete ignores error if the object doesn't exist
2017-06-15 16:27:19 +02:00
Alexander Neumann
dd6ce5f9d8
Remove backend.Closer, use ioutil.NopCloser() instead
2017-06-15 15:58:23 +02:00
Alexander Neumann
9a8301fc74
prune: Fix status string for narrow terminals
...
Closes #1030
2017-06-15 15:41:07 +02:00
Alexander Neumann
aabe2a0a30
Merge pull request #1002 from restic/test-codecov
...
Remove codecov config file
2017-06-15 15:09:50 +02:00
Alexander Neumann
c79fb6fcdd
prune: Delete repacked files as the very last step
2017-06-15 14:46:50 +02:00
Alexander Neumann
af9ba3be91
backend: Add IsNotExist
2017-06-15 13:40:27 +02:00
Alexander Neumann
6f24d038f8
prune: Only remove data after index has been uploaded
...
Closes #1032
2017-06-15 13:12:46 +02:00
Alexander Neumann
cf65893c4b
s3: Allow setting the number of retries for minio-go
...
https://github.com/restic/restic/issues/1013#issuecomment-307883970
2017-06-12 21:09:37 +02:00
Alexander Neumann
bd7d5a429f
Merge pull request #1025 from restic/fix-1013
...
s3: Switch back to high-level API for upload
2017-06-12 19:58:12 +02:00
Alexander Neumann
7b54f6e642
Add entry to CHANGELOG
2017-06-12 19:56:50 +02:00
Alexander Neumann
422c0dfb5e
s3: Exit test loop for minio server on success
2017-06-11 20:49:56 +02:00
Alexander Neumann
73b296918b
s3: Reorder debug messages
...
This way the semaphore token acquisition can be observed in the debug
log.
2017-06-11 20:49:53 +02:00
Alexander Neumann
907c201693
debug: Add version number to debug log
2017-06-11 20:48:46 +02:00
Alexander Neumann
58de8bf392
swift/rest: Reduce number of connections
2017-06-11 20:48:46 +02:00
Alexander Neumann
a89a7a783a
s3: Correct comment on the `connections` option
2017-06-11 20:48:46 +02:00
Alexander Neumann
c422010597
s3: Fix test
2017-06-11 20:48:46 +02:00
Alexander Neumann
08e1d9ffad
s3: Switch back to high-level API, limit connections
2017-06-11 20:48:42 +02:00
Alexander Neumann
a4e8dc3371
s3: Improve error message in debug log
2017-06-11 11:22:25 +02:00
Alexander Neumann
19da56a6ea
debug: Add log before panic()
2017-06-11 11:22:25 +02:00
Alexander Neumann
d3c06c39f9
debug: Fix EOF detection in HTTP transport
2017-06-11 11:22:25 +02:00
Alexander Neumann
6301620428
s3: Add more debug logs
2017-06-11 11:22:25 +02:00
Alexander Neumann
a6f157f346
Merge pull request #1024 from restic/remove-unused
...
Remove unused code/variables
2017-06-11 11:18:02 +02:00
Alexander Neumann
8d4417ec92
Remove unused code/variables
2017-06-11 09:29:53 +02:00
Alexander Neumann
0b55be2581
prune: Fix debug log
2017-06-10 22:16:42 +02:00
Alexander Neumann
88a59fd0ca
options: Handle uint
2017-06-10 21:07:10 +02:00
Alexander Neumann
539674614b
Merge pull request #1019 from restic/fix-1017
...
ls: Print names with percent correctly
2017-06-10 12:43:46 +02:00
Alexander Neumann
9d1b9157d4
ls: Print names with percent correctly
...
Closes #1017
2017-06-10 12:17:21 +02:00
Alexander Neumann
5f449045d2
Merge pull request #1003 from fwilhe/contributing-md-link
...
Fix relative link to CONTRIBUTING.md
2017-06-09 20:56:21 +02:00
Alexander Neumann
3e4d236751
Merge pull request #1010 from restic/update-minio-go
...
Update github.com/minio/minio-go
2017-06-09 20:55:49 +02:00
Alexander Neumann
4fe6593fbe
Merge pull request #1011 from restic/fix-1009
...
pack: Handle small files
2017-06-09 20:53:52 +02:00
Florian Wilhelm
635633379a
Fix link to CONTRIBUTING.md
2017-06-09 00:36:31 +02:00
Alexander Neumann
48fecd791d
pack: Handle more invalid header cases
2017-06-08 21:04:07 +02:00
Alexander Neumann
a325a20fb4
s3: Increase wait time for minio server
2017-06-08 20:50:56 +02:00
Alexander Neumann
1f0916b01b
Merge pull request #1004 from restic/add-migrate-s3
...
Add 'migrate' command, change s3 layout
2017-06-08 20:48:27 +02:00
Alexander Neumann
eb767ab15f
pack: Handle small files
2017-06-08 20:40:12 +02:00
Alexander Neumann
92c0aa3854
Merge pull request #998 from restic/fix-820
...
fuse: Add cache for blob sizes
2017-06-08 20:21:26 +02:00
Alexander Neumann
a61016cb55
Update github.com/minio/minio-go
2017-06-08 19:40:06 +02:00
Alexander Neumann
eb7ddd6e11
Add entry to CHANGELOG
2017-06-08 19:21:52 +02:00
Alexander Neumann
ff3d2e42f4
migrate: Be a bit more verbose
2017-06-08 19:19:45 +02:00
Alexander Neumann
1aab123b6c
Merge pull request #1008 from chaquotay/patch-1
...
Fixing tiny typo
2017-06-08 19:04:14 +02:00
Stephan Müller
d11f8d294f
Fixing tiny typo
2017-06-08 13:27:22 +02:00
Alexander Neumann
04ded881f6
s3: Change the default layout to "default"
2017-06-07 23:08:20 +02:00
Alexander Neumann
4f9bf5312b
Add migrate
...
This commits adds a 'migrate' command and a migration to move s3
repositories from the 's3legacy' to the 'default' layout.
2017-06-07 23:08:02 +02:00
Alexander Neumann
7cf8f59987
layout: Add String() and Name()
2017-06-07 21:59:41 +02:00
Alexander Neumann
b8b5c8e8c9
s3: Rename struct to Backend
2017-06-07 21:59:01 +02:00
Alexander Neumann
a46baf7685
s3: Remove cache
2017-06-07 20:51:45 +02:00
Alexander Neumann
f2a51aa37c
Add entry to CHANGELOG
2017-06-07 20:51:08 +02:00
Alexander Neumann
233eaf8ee9
fuse: Improve semantics of the blob size cache
...
Wrap it in a struct and add a Lookup() function to make clear that it
is only queried, not changed, so we don't have any race conditions.
2017-06-07 20:04:58 +02:00
Alexander Neumann
067be2c551
fuse: Add cache for blob sizes
...
Closes : #820
2017-06-07 20:04:15 +02:00
Alexander Neumann
550e1feaec
Merge pull request #999 from restic/backend-use-semaphore
...
backends: Use new semaphore
2017-06-07 19:48:32 +02:00
Alexander Neumann
f90ce23f30
Merge pull request #994 from restic/add-context
...
Add context.Context to the backend
2017-06-07 19:11:56 +02:00
Alexander Neumann
29f8f8fe68
Update github.com/kurin/blazer
...
Reduces cost-intensive list_files API calls.
2017-06-07 19:10:05 +02:00
Alexander Neumann
48c1e7b00d
Fix location tests
2017-06-06 21:12:38 +02:00
Alexander Neumann
2175ccedd2
Remove codecov config file
2017-06-06 21:02:19 +02:00
Alexander Neumann
d4e74f20aa
Add context to dump command
2017-06-06 00:37:25 +02:00
Alexander Neumann
aa5bc39311
swift: Use semaphore
2017-06-06 00:33:25 +02:00
Alexander Neumann
46049b4236
rest: Use semaphore
2017-06-06 00:26:29 +02:00
Alexander Neumann
683ebef6c6
s3: Use semaphore
2017-06-06 00:17:39 +02:00
Alexander Neumann
5010e95c23
Add error handling to semaphore
2017-06-06 00:17:21 +02:00
Alexander Neumann
46b7a270a6
Add context parameters to tests
2017-06-05 23:56:59 +02:00
Alexander Neumann
cf497c2728
Add context to restic packages
2017-06-04 14:35:14 +02:00
Alexander Neumann
16fcd07110
Add a Context to the backend
2017-06-04 14:02:44 +02:00
Alexander Neumann
a9a2798910
Merge pull request #993 from restic/improve-search-performance
...
Improve find
2017-06-04 12:44:29 +02:00
Alexander Neumann
9cd664caa3
Add entry to CHANGELOG
2017-06-04 11:50:38 +02:00
Alexander Neumann
a90e0c6595
find: Check trees only once
2017-06-04 11:42:40 +02:00
Alexander Neumann
7b5efaf7b0
find: Move functions to struct
2017-06-04 11:38:46 +02:00
Alexander Neumann
3b7ca4ac35
find: Improve debug log
2017-06-04 11:22:56 +02:00
Alexander Neumann
40a61b82ce
Merge pull request #978 from restic/add-backblaze-backend
...
Add Backblaze B2 backend
2017-06-03 14:54:04 +02:00
Alexander Neumann
028f43299a
Merge pull request #975 from restic/add-swift-backend
...
Add swift backend
2017-06-03 14:52:47 +02:00
Alexander Neumann
3a4727f0f5
Add entry to CHANGELOG.md
2017-06-03 14:28:29 +02:00
Alexander Neumann
fec89f95fb
Improve swift backend
2017-06-03 14:28:18 +02:00
Bartłomiej Święcki
5681d41f76
Implement OpenStack swift backend
...
This commit implements support for OpenStack swift
storage server, tested on OVH public cloud storage.
Special thanks to jayme-github <tuxnet@gmail.com>
who helped with the implementation.
2017-06-03 14:26:29 +02:00
Alexander Neumann
efd61d97ef
Vendor github.com/ncw/swift
2017-06-03 14:25:57 +02:00
Alexander Neumann
3ed56f2192
Add entry to CHANGELOG.md
2017-06-03 14:24:59 +02:00
Alexander Neumann
122462b9b1
Add Backblaze B2 backend
...
This is based on prior work by Joe Turgeon <arithmetric@gmail.com>
@arithmetric.
2017-06-03 14:24:59 +02:00
Alexander Neumann
2217b9277e
Vendor github.com/kurin/blazer
2017-06-03 14:24:59 +02:00
Alexander Neumann
b5e0e3631b
Addd nev version section
2017-06-03 14:10:28 +02:00
Alexander Neumann
be68e43871
Fix link
2017-06-02 22:08:04 +02:00
Alexander Neumann
f6034c0882
Merge pull request #990 from tmcarr/fix_readme_links
...
Fix the links in the readme to render in RST
2017-06-02 21:59:41 +02:00
Travis Carr
f693781bf0
Fix the links in the readme to render right.
2017-06-02 12:32:13 -07:00
Alexander Neumann
3ae9be987f
Add VERSION file for 0.6.1
2017-05-31 23:52:13 +02:00
Alexander Neumann
ec0975c388
Add VERSION file for
2017-05-31 23:51:02 +02:00
Alexander Neumann
c2ce484e93
Add version to CHANGELOG
2017-05-31 23:50:54 +02:00
Alexander Neumann
e5c7c314a7
Add section about reproducible build to README
...
In addition, the build script isn't needed any more.
2017-05-31 23:48:56 +02:00
Alexander Neumann
6d36dcd46e
Merge pull request #987 from Thor77/minor-doc-fix
...
[docs] Fix paragraph not indented correctly in #Autocomplete
2017-05-31 23:23:27 +02:00
Thor77
96c9ecd20e
Fix paragraph not indented correctly
2017-05-31 21:40:47 +02:00
Alexander Neumann
997be9a036
Remove PR
2017-05-31 21:34:18 +02:00
Alexander Neumann
31fd8e98b9
Add Entry to CHANGELOG
2017-05-31 21:33:45 +02:00
Alexander Neumann
aa0f874c8d
s3: Simplify IsNotExist()
2017-05-31 21:23:01 +02:00
Alexander Neumann
5c59484d2b
s3: Return only basename in List()
2017-05-31 21:22:55 +02:00
Alexander Neumann
fba6211c99
Merge pull request #986 from restic/fix-regression-985
...
Allow many idle connections per host
2017-05-31 20:49:50 +02:00
Alexander Neumann
a8386e7d71
Add entry to CHANGELOG
2017-05-31 19:53:54 +02:00
Alexander Neumann
04b262d8f1
Allow many idle connections per host
...
Closes #985
2017-05-31 19:39:19 +02:00
Alexander Neumann
4dbbc24a44
Update Go version
2017-05-30 23:05:13 +02:00
Alexander Neumann
725d50554a
Merge pull request #981 from restic/reproducible-builds
...
build.go: Strip temporary path, allow reproducible builds
2017-05-29 23:49:49 +02:00
Alexander Neumann
ed91cafce2
Add entry to CHANGELOG
2017-05-29 23:46:48 +02:00
Alexander Neumann
de48a5ac9c
build.go: Strip temporary path, allow reproducible builds
2017-05-29 23:27:25 +02:00
Alexander Neumann
1d167f4680
v0.6.0
...
-----BEGIN PGP SIGNATURE-----
iQJJBAABCAAzFiEEz48Y8oRFdZc/edThkaaGi9P3qQcFAlkscsgVHGFsZXhhbmRl
ckBidW1wZXJuLmRlAAoJEJGmhovT96kHG2kQAIO/0GAJzDBSW/GOZkmPv/PWdw5U
20E6LPr6LyRNkZPKhbGPWQdtijB7aVcLH521a2lFFloLqYqYdAlO+EihCK3Sbutc
eJmKVGQKoQHYpRqwoJYXQiRglfBcIZTc+z03CMOuhm15N57NgBGWxfUOiwcvAOFy
UVsq49wiPYnhyWp/Ntaq5ayECaXmk8Tne7wnFsahj0ROtD7kQidpMs1MFt/M11GH
32HCQQyHjhuLqFhFWsAgWieihOPqyP85rFs1jt84YLKZFyOkzie3cIn+8RbW2J+F
R0nKxFyk4d5Zu8CdhNGCN3uAzq9mFuLXKCbd3+5iSb7ImsKi/cKnhV4N3kz57ZVz
k81WYMH4D8RJGrTt3V+w3WmqH/FcxKNby2V77sXuqrmt3gnBBld9wdC3YrO5HVIF
CKoW43/Ey4A7zZCF+Scnpv9xVLVFS+imMiSw249PTzlJoS9Cnbz25QnqdRT0yq9D
Fl246YVpnfxI0kd5S3AfvtscIFwibiUgU0tFwSralsSf+sxXfs5xQ/ApM9BEXOtD
IRCfgIHu+AmEslBjGjpR70RyK20yibiP505lcMCwHL8MMbN/6iYlW4Qdy+2Wk/Il
tFdSbd8evslw0mk5PsMSIoF+kdeHKtI+fEwppCrGatHjRTp5c+NMM7DBenqDgPHA
Jr3CnNYaxlMf9Vc2
=PyZB
-----END PGP SIGNATURE-----
Merge tag 'v0.6.0'
v0.6.0
2017-05-29 21:35:27 +02:00
Alexander Neumann
efad7ee197
Add VERSION file for 0.6.0
2017-05-29 21:31:41 +02:00
Alexander Neumann
820c88ea73
Add VERSION file for 0.6.0
2017-05-29 21:12:52 +02:00
Alexander Neumann
e7f031c9b3
Merge pull request #976 from restic/backend-fixes
...
Misc fixes for the backend/test code
2017-05-28 13:30:56 +02:00
Alexander Neumann
f3f6924b61
backend/test: Loose requirement about early error
2017-05-28 13:06:27 +02:00
Alexander Neumann
c5244abad9
rest: Improve error messages
2017-05-28 12:33:47 +02:00
Alexander Neumann
1f5954e2c1
layout: Test DefaultLayout for empty path prefix
2017-05-28 12:33:47 +02:00
Alexander Neumann
e046a2a6da
sftp: Use path instead of filepath
2017-05-28 12:33:47 +02:00
Alexander Neumann
8395b53400
backend/test: Reduce verbosity in logs
2017-05-28 12:33:47 +02:00
Alexander Neumann
24ec14738d
backend/test: Skip offset == length test
2017-05-28 12:33:47 +02:00
Alexander Neumann
79477fdfe4
backend/test: Randomize test suite
2017-05-28 12:33:47 +02:00
Alexander Neumann
7ec0543af3
testing: Add id to error message in panic
2017-05-28 10:17:04 +02:00
Alexander Neumann
e73e3cb4ba
Merge pull request #974 from restic/remove-noninteractive-progress
...
Remove regular status printing for non terminals
2017-05-25 18:56:55 +02:00
Alexander Neumann
317d9c4559
Add entry to the changelog
2017-05-25 17:06:06 +02:00
Alexander Neumann
5247de552a
Remove regular status printing for non terminals
2017-05-25 17:03:48 +02:00
Alexander Neumann
37b107b90b
build script: Check for dirty work directory
2017-05-25 15:50:37 +02:00