rdelaage
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,337 Commits 13 Branches 44 Tags 75 MiB
Commit Graph

676 Commits

Author SHA1 Message Date
Alexander Neumann e3e3a8a695 local: Add layout tests 2017-04-02 20:01:40 +02:00
Alexander Neumann 24ebf95f33 local: Automatically detect layout 2017-04-02 20:01:40 +02:00
Alexander Neumann 54465c92cc layout: Allow passing in a default layout 2017-04-02 20:01:36 +02:00
Alexander Neumann f7c4b3a922 Fix layout detection 2017-04-02 19:18:03 +02:00
Alexander Neumann 50dfa64a54 Add layout name parser 2017-04-02 17:57:28 +02:00
Alexander Neumann c6b8ffbb61 Add layout auto detection 2017-04-02 17:25:22 +02:00
Alexander Neumann 3e81dcdfc2 Add cloud and s3 layout 2017-03-26 22:20:10 +02:00
Alexander Neumann 782b740c95 local: Remove unused code 2017-03-26 22:14:37 +02:00
Alexander Neumann 3fd6fa6f86 local: Use Layout for filename generation 2017-03-26 21:53:26 +02:00
Alexander Neumann 6a201f7962 backend: Add Layout 2017-03-26 21:52:49 +02:00
Alexander Neumann 80a864c52c test: Add TempDir() helper 2017-03-26 20:40:45 +02:00
Alexander Neumann c8eea49909 debug: Allow creating insecure repositories 
Uses low-security KDF parameters for scrypt(). Do not use in production!
 2017-03-26 10:55:08 +02:00
Alexander Neumann 719bb18316 Parse extended options 2017-03-25 17:52:16 +01:00
Alexander Neumann a8a7701f60 options: Add namespace to Apply() 2017-03-25 17:52:16 +01:00
Alexander Neumann 2924ebc124 options: Add Apply() 2017-03-25 17:52:16 +01:00
Alexander Neumann f587a5f4f0 options: fail when key is configured twice 2017-03-25 17:52:16 +01:00
Alexander Neumann 946b4f4b86 Add extended options via -o/--option 2017-03-25 17:52:12 +01:00
Alexander Neumann d0a5e86da1 Add free-form options parsing 2017-03-25 17:45:11 +01:00
Alexander Neumann 2e0b19f63f location: Move to backend/location 2017-03-25 17:45:11 +01:00
Alexander Neumann 9861f3d435 location: Reformat tests, use sub tests 2017-03-25 17:45:10 +01:00
Alexander Neumann 482a6e9840 local: Add Config struct 2017-03-25 17:45:10 +01:00
Alexander Neumann 1733b24f9d doc: Add section about how to change the sftp port 
Closes #895
Related to #785
 2017-03-25 09:07:50 +01:00
Alexander Neumann 6f76a6db66 rest: Make backend honor the REST protocol 2017-03-16 21:50:26 +01:00
trbs f1ba45723c introduce RWLock for caching obj.Stat() 2017-03-14 23:05:51 +01:00
trbs b523eef294 Cache size of last ReadAt on S3 for performance 
Each obj.Stat() call adds another request to the S3 endpoint
for some commands a lot of ReadAt calls are made for the same
object in S3. This patch essentially cuts the number of calls
to S3 in this case in half. Speeding up the progress and lowering
costs to S3.
 2017-03-14 14:02:50 +01:00
Pauline Middelink 642cd3bebf Fix restoring files as non-root user 
As we cannot reliably detect in advance if we can set ownership, permissions,
timestamps or ext attributes, execute ALL the requested changes before
returning the first error we found.

Report total errors at end of restore and stop printing entire stacktraces
where just the error message is sufficient.

Fixes #655
 2017-03-12 16:51:58 +01:00
Alexander Neumann 887e81188f Merge pull request #877 from middelink/fix-875 
Fix crash at restoring pre-existing hardlinked files
 2017-03-11 10:13:43 +01:00
Pauline Middelink b56e16acd0 Fix crash at restoring pre-existing hardlinked files 
Remove target file, ignore non existing file errors.

Small memory saving: Only keep inodes around for files with a link count > 1.
(We will/can never be asked to restore a hardlinked file with one of the
files having a link count of 1.)

Closes #836
 2017-03-11 00:06:58 +01:00
Pauline Middelink b75186533d Restore does not truncate files it restores 
Missing O_TRUNC when doing O_CREAT.

Closes #875
 2017-03-10 22:46:13 +01:00
Alexander Neumann 8958efba60 Merge pull request #871 from middelink/fix-869 
Refactor output of `find` to allow for json and normal output.
 2017-03-10 20:03:26 +01:00
Pauline Middelink a9707a5728 Refactor output of `find` to allow for json and normal output. 
Rather complicated solution becaused I wanted to retain the streaming
character of the output, which means for json I have to manually add
headers and footers per snapshot scanned + a list around the whole
set.

As the json ouput is now partly handcrafted, add proper testing to catch
unintentional changes to the output, making it non-json compliant.

Closes #869
 2017-03-10 16:23:58 +01:00
Alexander Neumann deddedd88f Merge pull request #872 from middelink/fix-787 
Suppress (non error) output of `forget` during -q
 2017-03-09 21:23:36 +01:00
Pauline Middelink b38294f236 Suppress (non error) output of `forget` during -q 
Closes #787
 2017-03-09 20:20:17 +01:00
Pauline Middelink 8a05de537f Refactor `prune` and `rebuild_index` 
Factor out and reuse `rebuildIndex()` in cmd_rebuild_index and cmd_prune.

Use contexts.
 2017-03-08 20:30:52 +01:00
Pauline Middelink 8a92687d9a Refactor `find` and `ls` commands 
Implement filtering by using `FindFilteredSnapshots()` to iterate over the snapshots

Refactor cmd_ls' `PrintNode()` into format.go, reuse its pretty printing in both `find`
and `ls` commands.

Use contexts.
 2017-03-08 20:29:31 +01:00
Pauline Middelink 3432e7edcd Refactor `tag` to use `FindFilteredSnapshots()` 2017-03-08 20:28:44 +01:00
Pauline Middelink 3c6c17abcd Refactor `forget` and `snapshots` command 
Implement filtering by using `FindFilteredSnapshots()` to iterate over the snapshots

Refactor cmd_snapshots' `PrintSnapshots()` so its pretty printing can be used from
both `forget` and `snapshots`.

Use contexts.
 2017-03-08 20:24:58 +01:00
Pauline Middelink 11d237c252 New helper function `FindFilteredSnapshots` to iterate over snapshots 
This helper function takes a set of filters and/or a list of snapshots
from the commandline. It returns a channel of *Snapshot.

When snapshot ids are given, they are checked for validity and their
corresponding Snapshots returned. The snapshot id "latest" is handled
special to return either the last snapshot (no filters) or the last
snapshot matching the filters.

When no arguments are given, the filters are applied over all available
snapshots and these are returned.
 2017-03-08 20:19:12 +01:00
Pauline Middelink 0f7b6ec5ac Adapt `key` command to context world. 2017-03-08 20:17:30 +01:00
Pauline Middelink b4526c4e6e Enable the use of `context` in restic 
Set up a cancelble context in global options, hook it into the ctrl-C handler
for proper cancel propegation.

Bump up minimal requirement for Go to version 1.7 in documentation
and test-build files.
 2017-03-08 20:12:16 +01:00
Pauline Middelink 3eaaa0f286 Correct some typo's in comments. 2017-03-08 20:09:24 +01:00
Pauline Middelink b1c8071163 Add filtering to `mount` command 2017-03-08 19:59:19 +01:00
Pauline Middelink 3468108d4c Implement --tag processing to `backup` and `restore` command 
Add `tags` argument to `FindLatestSnapshot()`
 2017-03-08 19:55:58 +01:00
Alexander Neumann 33c8dd4ee5 Merge pull request #867 from middelink/optimize 
Optimize List() pipeline
 2017-03-08 11:50:47 +01:00
Pauline Middelink 7c989ca487 Optimize List() pipeline 2017-03-07 20:58:36 +01:00
Pauline Middelink e9a2982ecd Add `--path` for snapshot filtering by path. 
Add `--group-by-tags` for grouping on host,tags,dirs instead of host,dirs.

Borrow the snapshot printing from cmd_snapshot.

Closes #841
 2017-03-07 19:28:42 +01:00
Alexander Neumann f5a55a81f7 Merge pull request #865 from restic/handle-empty-snapshots 
Refuse to create empty snapshots
 2017-03-07 11:21:48 +01:00
Alexander Neumann 340f2c80a0 Merge pull request #864 from restic/find-case-insensitive 
find: Add option to ignore case
 2017-03-07 11:20:33 +01:00
Alexander Neumann bb144436c7 Add test for empty snapshot 2017-03-07 11:17:15 +01:00
Alexander Neumann b50d3ba805 Refuse to create empty snapshots 
Closes #862
 2017-03-07 11:12:34 +01:00
Alexander Neumann 221bef48c0 find: Add option to ignore case 
Closes #859
 2017-03-07 10:58:09 +01:00
Pauline Middelink afcc1ba706 Create a helper function to get the terminal width 
Rationale: contain terminal access to one file.
 2017-03-06 11:23:00 +01:00
Alexander Neumann bf88a62a16 Merge pull request #858 from middelink/snapshot-layout 
Fix layout issue in cmd_snapshot "ascii art"
 2017-03-06 09:10:35 +01:00
Pauline Middelink 354e8ffb82 Fix layout issue in cmd_snapshot "ascii art" 
The layouter does not account for multi tags when determining the
need for ascii art.

36fd8178  2017-03-03 21:35:04  abuseio.polyware.nl    NL          /
                                                      A       └──

vs

36fd8178  2017-03-03 21:35:04  abuseio.polyware.nl    NL      ┌── /
                                                      A       └──
 2017-03-06 02:49:15 +01:00
Pauline Middelink e1c828be3e Fix SamePaths() and make it into a receiver function 
Add `HasPath(paths []string) bool` to Snapshot for testing if the
snapshot has at least the paths given to the function.

Reimplemented SamePaths(paths []string) so it does what the name implies,
compare if all given paths are in the snapshot.
 2017-03-06 02:21:58 +01:00
Pauline Middelink 1fa2313aef Snapshot: Add Original ID 
The Original ID is used when the snapshot is modified (e.g. by `tag`
command). Adjust integration testing to assert correctness.
 2017-03-05 19:55:23 +01:00
Pauline Middelink 26e266a951 Fix type of ID field in `cmd_snapshots` type Snapshot 2017-03-05 19:55:22 +01:00
Pauline Middelink 208edaa3d1 Snapshot: Add `AddTags()` and `RemoveTags()` 
Both prevent duplicate tags.
 2017-03-05 19:55:22 +01:00
Pauline Middelink f6a258b4a8 Add `tag`: Manipulate tags on existing snapshots 
Add integration testing.
 2017-03-05 19:55:22 +01:00
Pauline Middelink 45e9f35654 Make columns for host and tags size width dynamicly on their content. 2017-03-05 05:32:01 +01:00
Pauline Middelink edd5c8b44d Add integration test to make sure cmd_backup adds tags when required. 2017-03-05 05:24:40 +01:00
Pauline Middelink 7238a3ee89 Changed cmd_snapshots to be testable (no more using os.Stdout) 2017-03-05 05:24:11 +01:00
Alexander Neumann bbcab800c9 Merge pull request #850 from middelink/fix-848 
Add progressbar to repack and blob remove phases of prune cmd.
 2017-03-04 18:47:03 +01:00
Pauline Middelink 5564c78e53 English typo: rewriten > rewritten. 2017-03-04 17:43:58 +01:00
Pauline Middelink 792b81725e Add progressbar to repack and blob remove phases of prune cmd. 2017-03-04 17:38:34 +01:00
Pauline Middelink 1273c6f3d4 Display the proper amount of bytes we will be pruning from the repo. 2017-03-04 15:17:44 +01:00
Alexander Neumann 90bc187355 Merge pull request #840 from middelink/master 
Display absolute paths when displaying the output of ls and find.
 2017-03-03 12:09:20 +01:00
Pauline Middelink 5ecaaea90b Really use absolute pathnames, not all systems use /. 2017-03-03 11:14:39 +01:00
Pauline Middelink 039e81b04b Fix unit test, we need to check for absolute paths now. 2017-03-02 23:30:56 +01:00
Pauline Middelink efb4315a1e Display absolute paths when displaying the output of ls and find. 2017-03-02 22:41:11 +01:00
Alexander Neumann f53d33ba34 Make ArchiveReader a struct 2017-03-02 15:45:35 +01:00
Alexander Neumann 9b776dc7ab Use new Index implementation for rebuild-index 2017-03-02 15:23:59 +01:00
Alexander Neumann 7c92994f10 Clarify variable name 2017-03-02 14:52:18 +01:00
Alexander Neumann bf97cc7efa Allow filtering absolute paths 
Before, the restorer called the filter function with a relative path,
this prevented anchoring absolute patterns (which just never matched).
Now call the restore function with an absolute virtual path, starting at
the filepath separator.

Closes #834
 2017-03-02 14:50:54 +01:00
Alexander Neumann 31ff506309 Ignore empty lines in --files-from 
Closes #822
 2017-02-27 19:42:00 +01:00
Alexander Neumann 685f5ebbd1 Add `--prune` switch to `forget` 2017-02-21 10:58:30 +01:00
Alexander Neumann b8ce1b4e69 Correct archiver behavior in case of errors 2017-02-18 17:46:06 +01:00
Alexander Neumann 132afbe83b Correct error check for ENOTSUP, add errors.Wrap() 2017-02-18 14:36:37 +01:00
Alexander Neumann ef52d15edd Continue if extended attribute cannot be read 2017-02-18 14:35:11 +01:00
Alexander Neumann 6df2f9e5ba Add support for extended attributes on FreeBSD 2017-02-18 14:35:11 +01:00
Jaap Gordijn 49cae0904f Add support for extended attributes (e.g. ACL) 2017-02-16 11:44:51 +01:00
Alexander Neumann 40685a0e61 Merge pull request #803 from ibib/adds-json-support 
Adds JSON support for the snapshots command
 2017-02-16 11:26:14 +01:00
Alexander Neumann 4772a4986b Merge pull request #808 from restic/fix-807 
restore: Make sure buffer is large enough
 2017-02-15 19:58:03 +01:00
Alexander Neumann 50d066befb restore: Make sure buffer is large enough 2017-02-15 15:19:28 +01:00
Øystein Olsen c35e48291d Fix correct number of arguments for key command 2017-02-13 16:16:34 +01:00
Øystein Olsen 511278b66a Fix wrong description of rebuild-index command in help text 2017-02-13 16:07:29 +01:00
Øystein Olsen 514a11346d Add long description of list command in help text 2017-02-13 16:06:27 +01:00
Øystein Olsen 2eb75bb941 Consistently refer to 'the' instead of 'a' repository in help text 2017-02-13 16:05:25 +01:00
Øystein Olsen 9922ce97bf Use lowercase consistently in help text 2017-02-13 16:02:47 +01:00
ibib 3ed4127297 Adds JSON support for the snapshots command 2017-02-12 21:43:39 +01:00
Jaap Gordijn 3047702ded Correct hardlinks for fuse directories 2017-02-11 21:54:23 +01:00
Alexander Neumann 8c34eaad15 Improve error message 2017-02-11 14:28:15 +01:00
Alexander Neumann 0492eabff1 Improve error messages 2017-02-11 14:24:11 +01:00
Alexander Neumann 7797e084f9 checker: Pass on error loading an index 2017-02-11 14:22:14 +01:00
Alexander Neumann b40aa66985 errors: Add method Wrapf 2017-02-11 14:22:04 +01:00
Alexander Neumann 76c06c5f2a Add check for modified index 2017-02-11 14:13:58 +01:00
Alexander Neumann 83538c745a Merge pull request #792 from restic/fix-791 
s3: Increase MaxIdleConnsPerHost
 2017-02-11 11:20:20 +01:00
Alexander Neumann f266741f40 s3/rest: raise connection limit to 40 2017-02-11 10:40:51 +01:00
Alexander Neumann 4795a5c5d1 Merge pull request #795 from restic/fix-767 
fuse: Add options --allow-other and --allow-root
 2017-02-11 10:38:33 +01:00
Alexander Neumann c14cb62cd7 fuse: Add options --allow-other and --allow-root 
Closes #767
 2017-02-10 21:58:10 +01:00
Alexander Neumann b3ec01521e Merge pull request #794 from restic/fix-789 
Use non-formatting functions of errors for strings
 2017-02-10 21:42:59 +01:00
Alexander Neumann 6483df5ee4 Merge pull request #793 from restic/add-host-parameter 
backup: Add `--hostname` parameter
 2017-02-10 21:42:56 +01:00
Alexander Neumann 6275d69a36 fuse: Show link count 2017-02-10 21:16:48 +01:00
Alexander Neumann 6300c8df56 Merge pull request #763 from jgfrm/issue25 
Support hard links
 2017-02-10 20:58:39 +01:00
Jaap Gordijn 366bf4eb0c Support hard links 
Closes #152
 2017-02-10 20:58:19 +01:00
Alexander Neumann 21b358c742 backend tests: Always close reader 2017-02-10 20:49:46 +01:00
Alexander Neumann 7a0303f7ae s3: Make sure to return connection token 2017-02-10 20:49:37 +01:00
Alexander Neumann 64165ea4c8 s3: Hold connection semaphore until Close() 2017-02-10 20:17:52 +01:00
Alexander Neumann c8fc789393 Use non-formatting functions of errors for strings 
Commands used:

    $ gofmt -w -r 'errors.Fatalf(x) -> errors.Fatal(x)' src
    $ gofmt -w -r 'errors.Errorf(x) -> errors.New(x)' src

Closes #789
 2017-02-10 19:39:49 +01:00
Alexander Neumann f145e1de0f backup: Add `--hostname` parameter 2017-02-10 19:37:33 +01:00
Alexander Neumann 36dee7d892 s3: Increase MaxIdleConnsPerHost 2017-02-10 19:25:42 +01:00
Alexander Neumann 073edd914d Merge pull request #783 from opennota/master 
Fix some typos
 2017-02-09 10:43:27 +01:00
opennota 25e459659a Fix some typos 2017-02-09 06:43:10 +07:00
Alexander Neumann fdebb022e4 Make sure backups read from stdin have a file name 2017-02-08 22:37:02 +01:00
Alexander Neumann ed1739acbd Merge pull request #779 from restic/benchmark-checker 
checker: Reduce memory usage
 2017-02-07 11:03:49 +01:00
Alexander Neumann 1f81919d4a checker: Reduce memory usage 
benchmark              old bytes     new bytes     delta
    BenchmarkChecker-4     25551348      4288037       -83.22%
 2017-02-06 21:19:27 +01:00
Alexander Neumann 436b5dc20c Add Blob.String() 2017-02-06 19:50:27 +01:00
Alexander Neumann 0c867b21ff Add benchmark for checker 2017-02-06 19:34:40 +01:00
opennota 24acb09a2a Allow --files-from to take a dash for stdin (fixes #769) 2017-02-06 18:43:44 +07:00
Alexander Neumann 4ca134a41c prune: Close backend reader after download 
Closes #777
 2017-02-05 15:40:30 +01:00
Alexander Neumann aee58a8c17 Add more tests for reading backups from stdin 2017-02-04 16:38:33 +01:00
Alexander Neumann 22f3e21266 Start error message with a lower case character 2017-02-03 17:06:06 +01:00
Alexander Neumann 807fcf07d9 Return an error if password is not set for stdin 
Closes #770
 2017-02-03 15:53:07 +01:00
Alexander Neumann 79e198451c Don't set GOMAXPROCS 
This was a temporary fix for Go earlier than 1.5 to run code on all
avaialble cores. We don't need that any more since we require at least
Go 1.6.
 2017-02-02 10:18:07 +01:00
Alexander Neumann c7d60279f7 Merge pull request #762 from restic/fix-759 
Add 'index' to list of options for 'cat' command
 2017-01-30 18:26:32 +01:00
Alexander Neumann 7d49c65dd0 Merge pull request #761 from restic/fix-758 
Remove inconsistencies regarding the `cat` command
 2017-01-30 18:25:49 +01:00
Alexander Neumann abd44ca7c5 Add 'index' to list of options for 'cat' command 2017-01-30 10:53:17 +01:00
Alexander Neumann 44f4ff9d37 Remove 'tree' from help text for 'cat' command 2017-01-30 10:50:52 +01:00
Alexander Neumann 524ce01423 Remove 'cat tree' command 2017-01-30 10:48:50 +01:00
Alexander Neumann 41f59ffc78 Fix 'cat' command for tree blobs 2017-01-30 10:48:21 +01:00
Alexander Neumann d7f52fd7e5 Allow listing blobs 
Closes #756
 2017-01-30 10:28:17 +01:00
Alexander Neumann 1f9db97d33 Merge pull request #728 from cit/improve-snapshot-output 
Add box-drawing character to the snapshots command
 2017-01-29 11:00:13 +01:00
Alexander Neumann 31f6093513 Fix tests 2017-01-27 12:47:34 +01:00
Alexander Neumann b9bddeff39 Normalise the backend API 
This makes the following changes, before:

    type backend interface {
        // Test a boolean value whether a File with the name and type exists.
        Test(t FileType, name string) (bool, error)

        // Remove removes a File with type t and name.
        Remove(t FileType, name string) error
    }

After:

    type backend interface {
        // Test a boolean value whether a File with the name and type exists.
        Test(h Handle) (bool, error)

        // Remove removes a File with type t and name.
        Remove(h Handle) error
    }
 2017-01-26 22:02:22 +01:00
Florian Adamsky 54c2f622a4 Add box-drawing character to the snapshots command 
Remove underscore of variable name

Format code with gofmt

Change snapshot output according to the discussion
 2017-01-26 21:54:27 +01:00
Alexander Neumann 8d6fdb7a3e Fix packer manager test on Windows 2017-01-25 17:51:34 +01:00
Alexander Neumann dceaae33ed PackerManager: Remove unused Finalize() method 2017-01-25 17:12:06 +01:00
Alexander Neumann e8995b85b8 rest backend: Do not close the reader 
Closes #747
 2017-01-25 17:12:06 +01:00
Alexander Neumann 925a3cfad7 backend: Check that backends do not close the reader 2017-01-25 17:12:03 +01:00
Alexander Neumann c99a44b122 Fix Random() function 2017-01-25 13:26:16 +01:00
Alexander Neumann afc593676a fuse: Improve memory usage 
Discard blobs that aren't in use any more. This greatly reduces memory
usage and will probably only trigger on sequential read (e.g. for
restore via fuse).

Closes #480
 2017-01-24 12:38:44 +01:00
Alexander Neumann 0b982d3316 Add restore test with larger files 2017-01-24 11:51:21 +01:00
Alexander Neumann 26da14f315 fuse: improve tests 2017-01-24 11:42:50 +01:00
Alexander Neumann 2de7e03698 repository.LoadBlob: Read correct number of bytes 2017-01-24 11:42:50 +01:00
Alexander Neumann 3d2fbed55f Add test for LoadBlob 2017-01-24 11:42:50 +01:00
Alexander Neumann 0e445ec0f5 checker: Use TestRepository 2017-01-24 11:42:50 +01:00
Alexander Neumann 31055d88a5 Add debug messages 2017-01-24 11:42:42 +01:00
Alexander Neumann 0d125725bc Merge pull request #741 from restic/rework-backend-api 
Rework backend API
 2017-01-23 20:18:31 +01:00
Alexander Neumann 8b09b5b3cd Merge pull request #740 from restic/add-debug-profiles 
Add debug memory/cpu profile options
 2017-01-23 20:18:28 +01:00
Alexander Neumann 0d95507909 Fix test for PackerManager 2017-01-23 19:00:15 +01:00