Commit Graph

1389 Commits

Author SHA1 Message Date
Alexander Neumann 0005191d71 Remove `dirdiff` and `gentestdata` 2015-06-21 17:40:22 +02:00
Alexander Neumann 5ae04b6834 Add last integration tests, remove testsuite 2015-06-21 17:40:22 +02:00
Alexander Neumann 43d4558a90 Add test for backing up non-existing directories 2015-06-21 17:40:22 +02:00
Alexander Neumann 675f341b6d Output warnings/errors to configurable writer 2015-06-21 17:40:22 +02:00
Alexander Neumann cfaf8ab8a6 Add integration test for key handling 2015-06-21 17:40:22 +02:00
Alexander Neumann a99a460b32 Fix integration tests 2015-06-21 17:40:22 +02:00
Alexander Neumann a43733d552 Introduce `Verbosef` 2015-06-21 17:40:22 +02:00
Alexander Neumann 4388474cdc Restructure `cmd/restic`, no functional changes 2015-06-21 17:40:22 +02:00
Alexander Neumann 2fa259816b rename `opts` to `mainOpts` 2015-06-21 17:40:22 +02:00
Alexander Neumann 1216ded14b Add integration test for key command 2015-06-21 17:40:22 +02:00
Alexander Neumann e2563b3eca Fix comments 2015-06-21 17:40:22 +02:00
Alexander Neumann a176b1b5a6 Add more integration tests 2015-06-21 17:40:22 +02:00
Alexander Neumann 9853fbcf48 Remove more flags from tests 2015-06-21 17:40:21 +02:00
Alexander Neumann 3a65f27c3f Automatically find fixtures for integration test 2015-06-21 17:40:21 +02:00
Alexander Neumann 030f08a410 Remove flags from tests 2015-06-21 17:40:21 +02:00
Alexander Neumann d8d09b6d69 Fix restic configuration for integration tests 2015-06-21 17:40:21 +02:00
Alexander Neumann 12677b4f8a Use flag instead of build tag to run integration tests 2015-06-21 17:40:21 +02:00
Alexander Neumann 7c107acf0b More integration tests 2015-06-21 17:40:21 +02:00
Alexander Neumann 954025e399 Add integration test with the go testing framework 2015-06-21 17:40:21 +02:00
Chris Howey 47186806cf Add Amazon S3 backend. 2015-06-14 08:28:50 -05:00
Alexander Neumann 9c375ea382 Refactor backup a bit 2015-06-07 19:08:37 +02:00
Alexander Neumann fd80499954 Refactor terminal recognition, add --quiet parameter 2015-06-07 18:30:11 +02:00
Alexander Neumann f5537e7a0e Refactor configuration of cache dir and repository 2015-06-07 18:15:03 +02:00
Alexander Neumann 848793df84 Remove unused file 2015-06-07 18:13:22 +02:00
Alexander Neumann 0314d2b0c3 fsck: Fix check for orphaned blobs 2015-05-17 23:41:53 +02:00
Alexander Neumann df1e3e03e9 Add 'dump' command
'dump' writes internal data structes pretty-printed as JSON to stdout.
This was done to debug fsck error messages.

In contrast to the 'cat' command, this one prints the data structures as
there are interpreted by restic, not as they are stored in the
repository. This means that only the merged index from all the index
files is printed out.

This is meant for debugging only, it's compiled only when the "debug"
tag is active.
2015-05-17 23:15:08 +02:00
Alexander Neumann af381c2ab0 Remove Repository.Remove() 2015-05-17 20:58:22 +02:00
Alexander Neumann eb6dfcf58c Remove Repository.Test() 2015-05-17 20:58:22 +02:00
Alexander Neumann 6e38a8a033 Move FindSnapshot, make Repository.List() return IDs 2015-05-17 20:58:22 +02:00
Florian Weingarten bace4607bf Fix ModTime for directories 2015-05-13 23:11:31 -04:00
Alexander Neumann 95536e8a21 Rename 'Repo' -> 'Repository' 2015-05-09 23:59:58 +02:00
Alexander Neumann 232c472836 Move package 'repo' to package 'repository' 2015-05-09 23:52:03 +02:00
Alexander Neumann b2dcdf00e3 Merge pull request #178 from restic/refactor-server
Rename Server -> Repository
2015-05-09 22:59:02 +02:00
Alexander Neumann ae21938f3e Rename 'Repository' -> Repo 2015-05-09 17:41:28 +02:00
Alexander Neumann cd4c56b927 wip 2015-05-09 13:54:47 +02:00
Alexander Neumann d9b5832034 Rename variables 2015-05-09 13:47:21 +02:00
Alexander Neumann 87ebf12945 Rename 'Server' to 'Repository' 2015-05-09 13:25:52 +02:00
Alexander Neumann 8be9e95d20 Rename package 'server' to 'repo' 2015-05-09 13:24:15 +02:00
Alexander Neumann 861e1b9910 Add doc stubs to all the packages 2015-05-09 13:00:01 +02:00
Chris Howey e8615eb593 Fix ls command
Added missing loadindex call. Also fixed the recursive call to
use the subtree to stop infinte recursion.
2015-05-05 20:50:10 -05:00
Alexander Neumann bdbb3ab329 Remove pools.go 2015-05-05 00:14:07 +02:00
Alexander Neumann 35af933f24 server: Rename CreateMasterKey() to Init() 2015-05-04 20:40:17 +02:00
Alexander Neumann 1213d87b1a server: Only save crypto.Key
At the moment, the server doesn't need the full server.Key (master and
user key), just the master key.
2015-05-03 18:56:20 +02:00
Alexander Neumann 765e3dc66f restic: Add 'cat config' command 2015-05-03 18:56:20 +02:00
Alexander Neumann d4bf5bb279 server: Add config 2015-05-03 18:56:20 +02:00
Alexander Neumann bebb08ee7e server/key: Rename CreateKey -> CreateMasterKey 2015-05-03 18:56:20 +02:00
Alexander Neumann 9b54fd7bdb server: rename LoadJSONEncrypted -> LoadJSONUnpacked 2015-05-03 18:56:20 +02:00
Alexander Neumann b3a75ecc1c Correct usage for backup cmd 2015-05-02 02:05:09 +02:00
Florian Weingarten 0d9360a815 more cleanup 2015-04-29 22:33:34 -04:00
Alexander Neumann 24fc5d343f Remove leftover comment 2015-04-30 00:25:00 +02:00
Alexander Neumann 29ead95c96 Adapt fsck command to packed blobs (unfinished) 2015-04-29 23:27:36 +02:00
Alexander Neumann 60a0fe8349 Major restructure, bundle blobs
This commit also breaks the repository format.
2015-04-29 23:27:32 +02:00
Alexander Neumann d19b23d4f1 Move Server and Key to new sub-package 2015-04-26 14:46:15 +02:00
Alexander Neumann 6871e1bb58 Merge remote-tracking branch 'fw42/fix_nan_and_inf' 2015-04-26 12:09:14 +02:00
Florian Weingarten 61b1a5b1de Count errors separately and not as files that are "done" 2015-04-26 01:54:35 +00:00
Florian Weingarten 6c776468ab Fix floating point presentation 2015-04-25 13:20:41 -04:00
Alexander Neumann 87a1946417 Merge remote-tracking branch 'fw42/rename_change_to_passwd' 2015-04-25 11:11:27 +02:00
Alexander Neumann fd472cb59d Use short ID for init 2015-04-25 11:07:51 +02:00
Alexander Neumann 19486987b0 Merge remote-tracking branch 'fw42/mixed_caps' 2015-04-25 11:06:13 +02:00
Florian Weingarten dead2b7e93 rename "change" to "passwd" 2015-04-25 02:44:26 -04:00
Florian Weingarten 170d147427 Consistent writing 2015-04-24 19:43:27 -04:00
Florian Weingarten ebae478631 Prefer mixedCaps over underscore 2015-04-24 19:39:32 -04:00
Florian Weingarten 441f4deb66 remember your password 2015-04-24 19:08:05 -04:00
Matthieu Rakotojaona 1d0f8fa80e Take last snapshot as parent by default and allow force re-reading 2015-04-07 22:03:50 +02:00
Alexander Neumann 4d24f2eab3 Correct plen for snapshots 2015-03-28 16:16:19 +01:00
Alexander Neumann 223108c6ae Remove unused vars and const 2015-03-28 15:28:08 +01:00
Alexander Neumann f157f775da Add checks for Server.Load(), use Blob for load 2015-03-28 15:07:08 +01:00
Alexander Neumann 5e69788eac Refactor backends 2015-03-28 14:36:50 +01:00
Alexander Neumann 53ad706c6d Remove unused 'ram' tool 2015-03-22 21:03:25 +01:00
Alexander Neumann 0ed2a066a0 Document crypto and master key JSON struct 2015-03-22 21:03:01 +01:00
Alexander Neumann 5257c54585 Correct string for catting data blobs 2015-03-22 15:42:46 +01:00
Alexander Neumann 1c0e76ccd6 Bugfix: Backup changed data in incremental mode 2015-03-22 15:09:54 +01:00
Alexander Neumann cfa2229bc0 Refactor cache refresh for blobs, add progress 2015-03-22 15:09:50 +01:00
Alexander Neumann 865d315ad6 Shorten status output if terminal is small
Closes #107
2015-03-16 20:48:29 +01:00
Alexander Neumann 489cc654e8 Fix ETA display
Closes #111
2015-03-16 20:20:53 +01:00
Alexander Neumann b17840c6ee Refactor cache and 'cache' command 2015-03-14 12:57:29 +01:00
Alexander Neumann e1fc17aeb1 Implement per-repository cache 2015-03-14 12:16:28 +01:00
Alexander Neumann 14298fe232 Cache blobs for each snapshot 2015-03-09 23:50:32 +01:00
Alexander Neumann 798ac97a52 Re-enable dedup, add test (closes #95) 2015-03-07 12:05:33 +01:00
Alexander Neumann f167366be5 Add walk for trees, restructure 2015-03-02 19:58:06 +01:00
Alexander Neumann c8be54564f Add local cache 2015-03-02 10:04:01 +01:00
Alexander Neumann a59b0ec1f6 Add progress report for loading blobs 2015-02-21 15:32:48 +01:00
Alexander Neumann 58cded6b75 Refactor progress reporting 2015-02-21 14:25:06 +01:00
Alexander Neumann a0fea201d9 Reenable Preloading 2015-02-18 23:10:59 +01:00
Alexander Neumann 3bb2aba141 Remove test files 2015-02-18 21:50:59 +01:00
Alexander Neumann 65aae5f8c1 Disable preloading for now 2015-02-17 23:38:15 +01:00
Alexander Neumann b6f25aa690 Remove Each(), add basic stats 2015-02-17 23:11:56 +01:00
Alexander Neumann 4bb7f2f2ed Add Preloader for blobs 2015-02-17 22:39:44 +01:00
Alexander Neumann 4d1e7b9f24 Change parameters of LoadTree() 2015-02-17 22:37:34 +01:00
Alexander Neumann d4cfa7b797 Remove scanner (not needed any more) 2015-02-16 23:47:14 +01:00
Alexander Neumann f8f8107d55 wip 2015-02-15 15:17:24 +01:00
Alexander Neumann c884704bce Refactor pool stats 2015-02-08 13:46:51 +01:00
Alexander Neumann 57b8373bc9 Progress: Rename functions, fix documentation 2015-02-07 15:27:09 +01:00
Alexander Neumann 316e761f27 fsck: Check minimal set of blobs 2015-02-04 22:33:48 +01:00
Alexander Neumann 84ad6cf4d2 Return error on backup 2015-02-03 22:05:46 +01:00
Alexander Neumann fb95f02af6 Refactor testsuite 2015-01-17 16:32:24 +01:00
Alexander Neumann 25a214809b Refactor debug into debug module 2015-01-17 16:32:13 +01:00
Alexander Neumann bcb19c811b Add 'version' command 2015-01-16 21:26:33 +01:00
Alexander Neumann 7af11eaaae Refactor tests, integrate debug break points 2015-01-14 17:32:37 +01:00
Alexander Neumann 203a911de9 Refactor repository structure
Merge Map data type into Tree.
2015-01-14 16:34:30 +01:00
Alexander Neumann 33bcf31bae Fix incremental backup
Copying blobs (for unchanged files and subtrees) from old BlobList to
new BlobList was missing
2015-01-05 21:40:43 +01:00
Alexander Neumann 4b070358ef fsck: more verbose error messages 2015-01-05 20:38:49 +01:00
Alexander Neumann 6eb969a492 Update modified files, store error message 2015-01-04 23:25:13 +01:00
Alexander Neumann e543f5926c Refactor Archiver and backup command
Improve incremental backup by first scanning the tree, loading the old
tree and afterwards comparing both trees in memory.
2015-01-04 22:39:30 +01:00
Alexander Neumann fe231af7fc Remove error from return value of NewContentHandler 2015-01-04 20:07:30 +01:00
Alexander Neumann 4b70bba588 Split Scanner from Archiver, refactor Progress 2015-01-04 18:23:00 +01:00
Alexander Neumann a93bc3c991 Rename functions, remove code for compare
* Archiver: Rename loadTree -> scan and LoadTree -> Scan
 * Archiver: Remove code to compare against an old snapshot/tree,
   the current implementation was really slow
2015-01-04 16:13:50 +01:00
Alexander Neumann 5b82475d74 Merge branch 'restore-pattern', closes #69 2015-01-02 23:04:29 +01:00
Sebastian Schmidt b1dbc6f062 Support for pattern in restore command
This just matches the passed pattern against the full source path with
filepath.Match which, in contrast go filepath.Glob, doesn't match the
directory separator with '*' and is not terribly useful that way.
Someone should replace that by a more sophisticated matcher.
2015-01-02 23:03:07 +01:00
Alexander Neumann 2e28b88e3a Write prompt to stderr instead of stdout 2015-01-02 22:36:53 +01:00
Sebastian Schmidt 190b1cdcc7 Fix 'ls' command
CmdLs needs to implement flags.Commander, so change (CmdLs) Execute to
only take a []string to get actually called.
2015-01-01 11:30:30 +01:00
Alexander Neumann 1944ab13d4 Add dangling blob detection and removal to 'fsck' 2014-12-22 14:46:54 +01:00
Alexander Neumann a981141e49 Rename Table.Print -> Table.Write 2014-12-22 11:21:14 +01:00
Alexander Neumann 5431b025a3 Reduce code duplication in key handling 2014-12-21 18:16:22 +01:00
Alexander Neumann ef41a77aff Remove explicit key handling 2014-12-21 18:10:19 +01:00
Alexander Neumann cc147c002e Introduce type Server 2014-12-21 17:37:29 +01:00
Alexander Neumann 0e1045301a Optimize 'cat' and 'snapshots' commands 2014-12-21 17:29:37 +01:00
Alexander Neumann 879abd0d12 Add time to command 'find' 2014-12-07 17:23:00 +01:00
Alexander Neumann 28bb061ad3 Refactor commands 2014-12-07 16:32:23 +01:00
Alexander Neumann b3deca33a9 Add command 'find' 2014-12-07 14:44:01 +01:00
Alexander Neumann 988d18bb1d Improve output 2014-12-07 14:26:32 +01:00
Alexander Neumann 202984a93f Rename snapshot.Content -> snapshot.Tree 2014-12-07 14:14:50 +01:00
Alexander Neumann 45e40eb27a Correctly encode non utf8 node names 2014-12-07 13:06:35 +01:00
Alexander Neumann fa94d408f3 Remove old khepri binary 2014-12-06 17:59:21 +01:00
Alexander Neumann e2fea0d088 Rename khepri -> restic 2014-12-05 21:45:49 +01:00
Alexander Neumann 0ac1716463 Fix error message for invalid snapshot id in `fsck` 2014-11-30 22:49:41 +01:00
Alexander Neumann 172b7bf123 Move command init into cmd_* files 2014-11-30 22:39:58 +01:00
Alexander Neumann bb8c52a974 Add 'fsck' command 2014-11-30 22:35:42 +01:00
Alexander Neumann 3325fa07ea Check timestamps for files 2014-11-30 22:34:21 +01:00
Alexander Neumann 65c9a78e95 Remove extra linefeed from table header in 'snapshots' 2014-11-28 00:01:56 +01:00
Alexander Neumann 3e2c9dc267 Show currently used key in 'key list' 2014-11-27 23:26:19 +01:00
Alexander Neumann 3b243ed1dd Add command "key change" to change repository password 2014-11-25 23:23:09 +01:00
Alexander Neumann 43ccee3b08 Add command "key rm" 2014-11-25 23:18:02 +01:00
Alexander Neumann f7fcd7c04c Add command "key add" 2014-11-25 23:07:00 +01:00
Alexander Neumann d0a63cd49f Add command "key list" 2014-11-25 22:52:53 +01:00
Alexander Neumann ad7e1d043f Add simple ascii table implementation 2014-11-25 22:39:35 +01:00
Alexander Neumann bb62218743 Add functions to compute unique prefix length 2014-11-24 22:09:22 +01:00
Alexander Neumann 2c5d07a571 List snapshots, accept snapshot id prefix
Example:

    $ ./khepri snapshots
    ID        Date                 Source      Directory
    --------------------------------------------------------------------------------
    fa31d65b  2014-11-24 19:45:11  kasimir     /home/user/testdata
    20bdc140  2014-11-24 20:00:47  kasimir     /home/user/testdata
    326cb59d  2014-11-24 20:01:40  kasimir     /home/user/testdata
    20ff988b  2014-11-24 20:35:35  kasimir     /home/user
2014-11-24 21:12:32 +01:00
Alexander Neumann 339c63befc Add "list maps" command 2014-11-23 23:29:29 +01:00
Alexander Neumann 725e5bfca5 Require repository 2014-11-23 23:19:47 +01:00
Alexander Neumann 33ce4e33aa Store maps in new subdir "maps" 2014-11-23 22:26:01 +01:00
Alexander Neumann f95788ed90 Limit memory usage, add several sync.Pool 2014-11-23 21:26:20 +01:00
Alexander Neumann 575635753e Correct wildcard in Makefile 2014-11-23 15:15:11 +01:00
Alexander Neumann a780462b80 Change bps and eta calculation to prevent overflow 2014-11-23 14:56:41 +01:00
Alexander Neumann 6d0a6844e2 Add space between number and unit 2014-11-23 14:54:37 +01:00
Alexander Neumann 3f0ea78b86 Clear line with ANSI escape on update 2014-11-23 14:34:18 +01:00
Alexander Neumann 7e26567b8b Pretty status for backup 2014-11-23 13:51:15 +01:00
Alexander Neumann 21df0e50e5 Refactor stats
* channel instead of callback func
 * cumulate Stats for slow receivers
2014-11-23 13:51:02 +01:00
Alexander Neumann 7d1ba8ab65 Ignore permission errors for backup 2014-11-23 12:15:19 +01:00
Alexander Neumann 94d1482888 Save multiple files in parallel 2014-11-18 21:46:44 +01:00
Alexander Neumann 4a3a6861e2 Scan directory first, then backup. Add stats. 2014-11-17 23:52:31 +01:00
Alexander Neumann 616a2c749d Use different tags for debug log 2014-11-17 23:52:31 +01:00
Alexander Neumann cf33b01582 Load config.mk in Makefile 2014-11-17 23:52:31 +01:00
Alexander Neumann 7006e13ca9 Add dependencies 2014-11-17 23:52:31 +01:00
Alexander Neumann bd105b69a8 Add debug() function to khepri cmd and lib 2014-11-17 23:52:31 +01:00
Alexander Neumann 804cebde67 Rename backend subdir 'blobs' to 'data' 2014-11-16 13:22:19 +01:00
Alexander Neumann ff3d0840d8 Fix integration tests
Todo for later:
 * fix Futimes() syscall (will be done in #24)
 * return error in dirdiff when ModTime doesn't match
2014-11-16 12:05:54 +01:00
Alexander Neumann d20c4ff316 Correctly output blob after decryption with cat 2014-11-15 20:19:34 +01:00
Alexander Neumann 8277daa9e1 Add Makefile and version.sh, add 'version' command 2014-11-15 19:08:15 +01:00
Alexander Neumann ebd4f97350 Update import path 2014-11-15 19:04:32 +01:00
Alexander Neumann bb5bfe1ba0 Cleanup, remove unused commands 2014-10-05 16:17:40 +02:00
Alexander Neumann 09702c1c61 Add commands 'cat','list' and 'ls' 2014-10-05 14:44:59 +02:00
Alexander Neumann f848afed27 Add SFTP backend 2014-10-04 19:20:15 +02:00
Alexander Neumann 30ab03b7b7 Add decrypt, refactor 2014-10-03 21:44:55 +02:00
Alexander Neumann 03ca69407d Add method to create repository
Also disables automatic creation on open
2014-09-23 21:16:54 +02:00
Alexander Neumann f0287b2c9a Restructure, adapt for max 16MB chunk size 2014-09-21 16:33:20 +02:00
Alexander Neumann f1f96c4206 Remove old cmd/ experiments 2014-09-21 16:02:57 +02:00
Alexander Neumann 0a4c9aab36 Add error to messages 2014-09-21 15:58:52 +02:00
Alexander Neumann 16dfd73ecb Extend cmd/split 2014-09-18 22:40:59 +02:00
Alexander Neumann 18131f88bf Add code for content defined chunking (CDC) 2014-09-18 21:10:20 +02:00
Alexander Neumann 40aea3d612 Remove temp dir 2014-08-12 21:39:14 +02:00
Alexander Neumann d66996e648 Move restore functionality into khepri package 2014-08-11 23:14:40 +02:00
Alexander Neumann 2428843faa Refactor 2014-08-11 22:47:24 +02:00
Alexander Neumann d60828fc15 Add dirdiff to tests 2014-08-06 20:09:51 +02:00
Alexander Neumann 017426467b Refactor Makefiles 2014-08-06 20:09:41 +02:00
Alexander Neumann 16d7974b3f Add dirdiff command 2014-08-06 20:09:24 +02:00
Alexander Neumann ab602b88de Add Makefiles and simple test suite 2014-08-05 23:13:19 +02:00
Alexander Neumann f45a9c8e9f Fix logging 2014-08-05 23:13:07 +02:00
Alexander Neumann ad8125d83c Add command 'fsck' 2014-08-04 23:25:58 +02:00
Alexander Neumann b923a2065a Rename field in Snapshot: Tree -> TreeID 2014-08-04 23:25:32 +02:00
Alexander Neumann 500f4f9997 Add command 'snapshots' 2014-08-04 22:55:54 +02:00
Alexander Neumann e8b83e460f Refactor 2014-08-04 22:46:14 +02:00
Alexander Neumann 022f514b09 Rename ListIDs -> List 2014-08-04 22:15:04 +02:00
Alexander Neumann e525655dcb Fix backup command 2014-08-04 21:53:25 +02:00
Alexander Neumann b3c2d82331 Refactor, add Object and Snapshot 2014-08-04 20:51:09 +02:00
Alexander Neumann fbd33636f0 Rename types and errmsg function 2014-08-03 16:36:52 +02:00
Alexander Neumann 5cbd1d0090 Add command 'list' 2014-08-01 22:20:28 +02:00
Alexander Neumann 11d6aa5328 Fix commands 2014-08-01 22:20:15 +02:00
Alexander Neumann 13bb557cdc Restructured repository 2014-07-28 20:21:19 +02:00