Alexander Neumann
90c1608d88
sftp: Add Benchmarks
2017-05-14 11:48:30 +02:00
Alexander Neumann
7a51640262
rest: Add benchmarks
2017-05-14 11:48:30 +02:00
Alexander Neumann
13946e7db7
mem: Add benchmarks
2017-05-14 11:48:30 +02:00
Alexander Neumann
e009c002ba
local: Add benchmark
2017-05-14 11:48:30 +02:00
Alexander Neumann
d24e0cc6cc
s3: Add benchmarks
2017-05-14 11:48:30 +02:00
Alexander Neumann
5b8131e2d3
tests: Add benchmark for Save
2017-05-14 11:48:30 +02:00
Alexander Neumann
8fc25cc567
tests: Add benchmarks for partial file (+offset)
2017-05-14 11:48:30 +02:00
Alexander Neumann
77ebb95d3d
tests: Add BenchmarkLoadFile
2017-05-14 11:48:30 +02:00
Alexander Neumann
f142b1c22f
tests: Add benchmarks to test suite
2017-05-14 11:48:30 +02:00
Alexander Neumann
ee68f9298b
tests: Ensure that backend tests cannot be skipped on Travis
2017-05-14 00:33:15 +02:00
Alexander Neumann
5c6ec78789
s3: Remove log line
2017-05-13 19:58:25 +02:00
Alexander Neumann
4ac0d3ad40
backend tests load: Use reader with Size() method
2017-05-13 19:56:11 +02:00
Alexander Neumann
ec5e984ed9
Increase waiting time for network services
2017-05-12 21:45:18 +02:00
Alexander Neumann
88de3cfecc
rest: Wait until REST server is started
2017-05-12 21:33:34 +02:00
Alexander Neumann
a654f41ddb
Add docstring for package test
2017-05-12 21:19:32 +02:00
Alexander Neumann
fbf2462325
Rename backend test functions
2017-05-12 21:06:49 +02:00
Alexander Neumann
59d1986660
s3: Use random prefix for tests
2017-05-12 21:04:04 +02:00
Alexander Neumann
db4fa48f66
rest: Add context
2017-05-12 21:04:04 +02:00
Alexander Neumann
f2b83ece10
s3: Ignore errors for terminated minio server
2017-05-12 21:04:04 +02:00
Alexander Neumann
860b52273e
s3: Add tests for AWS
2017-05-12 21:04:04 +02:00
Alexander Neumann
bafceb56fb
s3: Convert to test suite, run Minio automatically
2017-05-12 21:04:04 +02:00
Alexander Neumann
9271b3662a
sftp: ignore error
2017-05-12 21:04:04 +02:00
Alexander Neumann
554013ca9f
rest: Convert to test suite
2017-05-12 21:04:04 +02:00
Alexander Neumann
3250fdc2ca
mem: Convert to test suite
2017-05-12 21:04:04 +02:00
Alexander Neumann
3678d34b16
sftp: Convert to test suite
2017-05-12 21:04:04 +02:00
Alexander Neumann
16276853a1
local: Convert to test suite
2017-05-12 21:04:04 +02:00
Alexander Neumann
f10c24e404
Improve test helpers
2017-05-12 21:04:04 +02:00
Alexander Neumann
6f5fd72738
Implement test suite for backend tests
2017-05-12 21:03:55 +02:00
Pauline Middelink
103a491ac0
Make houndci-bot happy.
2017-05-11 22:38:12 +02:00
Pauline Middelink
929f90344e
Change backup policy to be inclusive, meaning all given policies
...
are evaluated for each snapshot, thereby making sure that each
keep-* is able to retain its most recent snapshot. Thereby insuring
that weeklies keep Sundays around and monthlies keep the last day of
the month around.
Added testcase to make sure when multiple --keep-tags are given,
ALL of them need to match.
2017-05-11 22:34:50 +02:00
Alexander Neumann
a963052d64
Add custom HTTP transport
2017-05-11 21:51:30 +02:00
Alexander Neumann
898613e14f
debug: Add RoundTripper
2017-05-11 21:51:30 +02:00
Alexander Neumann
b84e63d503
backend.LoadAll: Check errors from io.Copy()
2017-05-11 21:51:30 +02:00
Alexander Neumann
63870d2830
Check error for Close()
2017-05-11 21:51:30 +02:00
Alexander Neumann
54e46f5984
Add MinimalData option for backend tests
2017-05-11 21:51:30 +02:00
Alexander Neumann
0096eca7fe
Install log buffer for global logs
...
This hides logged message from the net/http library and only shows them
in case an error occurs.
2017-05-11 21:51:30 +02:00
Pauline Middelink
ce3acbd30a
Remove unused SnapshotFilter and FilterSnapshots().
2017-05-11 21:07:58 +02:00
Alexander Neumann
edbd6ad584
Add fs.TempFile and fs.RemoveIfExists
2017-05-10 19:48:22 +02:00
Alexander Neumann
bcfe7afbfd
Merge pull request #945 from middelink/fix-942
...
Add bash autocompletion generation
2017-05-07 10:30:10 +02:00
Alexander Neumann
46e6e8984d
Merge pull request #941 from restic/improve-mkdir
...
local/sftp: Auto create directories if needed
2017-05-07 10:26:39 +02:00
Pauline Middelink
2c42629c51
Add bash autocompletion generation
...
Fix #942
2017-05-06 12:39:23 +02:00
Pauline Middelink
f93e5a39e5
Small textual changes to apply, to reflect actual meaning.
...
Added tests for the ExpirePolicy operations.
2017-05-04 16:39:41 +02:00
Pauline Middelink
02eafe94b3
Use global context in cmd_forget, like all other cmds
2017-05-04 16:35:35 +02:00
Alexander Neumann
7d914768bb
local/sftp: Auto create directories if needed
...
As reported in #940
2017-05-03 21:19:22 +02:00
Alexander Neumann
548d4eed95
Correct backend test for len = 0
2017-04-26 20:47:15 +02:00
Alexander Neumann
ce28584dda
Don't sort flags for `forget` command
2017-04-21 19:25:21 +02:00
Alexander Neumann
d0db1bf9b3
Test if a config file on open
2017-04-19 18:56:01 +02:00
Alexander Neumann
e9d939ff2d
Update code for pkg/xattr v0.2.0
2017-04-18 21:39:55 +02:00
Alexander Neumann
28968caf33
s3: Correct prefix for layout
2017-04-17 20:06:55 +02:00
Alexander Neumann
16fd1c2352
s3: Correct layout handle url/path
2017-04-17 20:06:55 +02:00
Alexander Neumann
b942f61272
s3: Reduce connection limit, prevents timeouts
2017-04-17 20:06:55 +02:00
Alexander Neumann
69dd1d2544
backup: Allow specifying multiple exclude files
2017-04-16 20:52:41 +02:00
Alexander Neumann
dd65ac56ef
filter: Ignore empty patterns
2017-04-16 20:49:20 +02:00
Alexander Neumann
c796d84fca
Ignore empty lines in excludes file
...
Closes #915
2017-04-16 20:46:52 +02:00
Alexander Neumann
525db875b0
Merge pull request #898 from restic/prepare-cloud-backends
...
Prepare more cloud backends, add backend layouts
2017-04-15 11:30:25 +02:00
Alexander Neumann
5bd95b3ce1
Implement MkdirAll() for Windows
...
Closes #735
2017-04-15 10:56:17 +02:00
Alexander Neumann
be06983c80
options: Fix sorting (and test)
2017-04-14 00:45:54 +02:00
Alexander Neumann
a634c22ae0
Add hidden 'options' command to list all opts
2017-04-14 00:00:03 +02:00
Alexander Neumann
859ee23d2e
options: Register local and sftp backends
2017-04-14 00:00:03 +02:00
Alexander Neumann
b7671dafc8
options: Allow registering
2017-04-14 00:00:02 +02:00
Alexander Neumann
541484d142
s3: Use Layout
2017-04-14 00:00:02 +02:00
Alexander Neumann
f531ca3b48
layout: Fix corner cases
2017-04-14 00:00:02 +02:00
Alexander Neumann
0da7264e75
rest: Convert to Layout
2017-04-14 00:00:02 +02:00
Alexander Neumann
7f3bcdb4cc
layout: prepare use for REST backend
2017-04-14 00:00:02 +02:00
Alexander Neumann
5eaa51eeff
Remove unused assignments
2017-04-14 00:00:02 +02:00
Alexander Neumann
7b64b890d7
Simplify code
2017-04-14 00:00:02 +02:00
Alexander Neumann
ccc201ea5f
remove unused code
2017-04-14 00:00:02 +02:00
Alexander Neumann
e6578857cf
sftp/local: Fix listing files
2017-04-14 00:00:02 +02:00
Alexander Neumann
320c22f1f5
backend/layout: Add Basedir()
2017-04-14 00:00:02 +02:00
Alexander Neumann
e2af5890f3
backend: Add test for listing files with layouts
2017-04-14 00:00:02 +02:00
Alexander Neumann
36b1c0898c
sftp: Add OS X sftp-server path
2017-04-13 23:59:59 +02:00
Alexander Neumann
c723cdf808
Check allowed devices per path
...
Closes #645
Closes #702
2017-04-13 21:03:05 +02:00
Alexander Neumann
c2ee0d9c84
sftp: Skip tests if server binary is not available
2017-04-10 22:51:00 +02:00
Alexander Neumann
e8780f1ec6
sftp: Add layout tests
2017-04-10 22:42:33 +02:00
Alexander Neumann
783fd73ea1
local: Rename local_layout_test
2017-04-10 22:42:33 +02:00
Alexander Neumann
74eb293733
sftp: Remove legacy filename/dirname methods
2017-04-10 22:42:33 +02:00
Alexander Neumann
42ea4d257b
sftp: first step of conversion to Layout
2017-04-10 22:42:33 +02:00
Alexander Neumann
a849edf19a
local: remove double Close()
2017-04-10 22:42:33 +02:00
Alexander Neumann
0cbd59856c
layout: Add IsNotExist
2017-04-10 22:42:33 +02:00
Alexander Neumann
698ba57597
backend/tests: Print error stacktrace if available
2017-04-10 22:42:33 +02:00
Alexander Neumann
27ce6a85e9
sftp: Rework Open/Create
2017-04-10 22:42:33 +02:00
Alexander Neumann
ae290ab374
sftp: Rename Dir -> Path
2017-04-10 22:42:33 +02:00
Alexander Neumann
ab602c9d14
sftp: Add Layout
2017-04-10 22:40:24 +02:00
Alexander Neumann
2e53af1b75
sftp: Rename Open/Create
2017-04-10 22:39:13 +02:00
Alexander Neumann
a725e065d9
Merge pull request #903 from restic/fix-hardlinks
...
Add new field DeviceID and tests
2017-04-07 21:05:20 +02:00
Alexander Neumann
db7e23b423
Skip /dev/null on darwin
2017-04-07 20:37:20 +02:00
Alexander Neumann
10a395ca33
Make tests runnable on os x/darwin
2017-04-06 20:36:09 +02:00
Alexander Neumann
522c7ade91
Add test for double defined flags
2017-04-06 19:44:53 +02:00
Alexander Neumann
a3d6099892
Remove short option conflict
2017-04-06 19:14:38 +02:00
Alexander Neumann
280028290e
Disable tests on darwin
2017-04-05 21:35:29 +02:00
Alexander Neumann
c195139d31
Only run tests on unix
2017-04-05 20:51:26 +02:00
Alexander Neumann
6f1b03415c
Fix hardlinks
2017-04-05 20:45:24 +02:00
Alexander Neumann
ffcb015581
Add new field DeviceID and tests
2017-04-05 20:42:15 +02:00
jgfrm
6674b2a70c
Moved assignment of device id outside the switch
2017-04-05 11:33:08 +02:00
jgfrm
c855d6bb9a
storage of device id of files
2017-04-04 21:28:25 +02:00
Alexander Neumann
1086528ab7
sftp: Fix errors import
2017-04-03 21:42:41 +02:00
Alexander Neumann
c26dd6b76f
sftp: Integrate command
2017-04-03 21:05:42 +02:00
Alexander Neumann
d3b6f75848
sftp: Add SplitShellArgs
2017-04-03 08:57:33 +02:00
Alexander Neumann
d1efdcd78e
Add integration test for layouts
2017-04-02 20:35:17 +02:00
Alexander Neumann
95ab5adda1
local: Expose layout as extended option
2017-04-02 20:29:00 +02:00
Alexander Neumann
c5eb36fe9d
layout: improve error message for ParseLayout
2017-04-02 20:28:42 +02:00
Alexander Neumann
e3e3a8a695
local: Add layout tests
2017-04-02 20:01:40 +02:00
Alexander Neumann
24ebf95f33
local: Automatically detect layout
2017-04-02 20:01:40 +02:00
Alexander Neumann
54465c92cc
layout: Allow passing in a default layout
2017-04-02 20:01:36 +02:00
Alexander Neumann
f7c4b3a922
Fix layout detection
2017-04-02 19:18:03 +02:00
Alexander Neumann
50dfa64a54
Add layout name parser
2017-04-02 17:57:28 +02:00
Alexander Neumann
c6b8ffbb61
Add layout auto detection
2017-04-02 17:25:22 +02:00
Alexander Neumann
3e81dcdfc2
Add cloud and s3 layout
2017-03-26 22:20:10 +02:00
Alexander Neumann
782b740c95
local: Remove unused code
2017-03-26 22:14:37 +02:00
Alexander Neumann
3fd6fa6f86
local: Use Layout for filename generation
2017-03-26 21:53:26 +02:00
Alexander Neumann
6a201f7962
backend: Add Layout
2017-03-26 21:52:49 +02:00
Alexander Neumann
80a864c52c
test: Add TempDir() helper
2017-03-26 20:40:45 +02:00
Alexander Neumann
c8eea49909
debug: Allow creating insecure repositories
...
Uses low-security KDF parameters for scrypt(). Do not use in production!
2017-03-26 10:55:08 +02:00
Alexander Neumann
719bb18316
Parse extended options
2017-03-25 17:52:16 +01:00
Alexander Neumann
a8a7701f60
options: Add namespace to Apply()
2017-03-25 17:52:16 +01:00
Alexander Neumann
2924ebc124
options: Add Apply()
2017-03-25 17:52:16 +01:00
Alexander Neumann
f587a5f4f0
options: fail when key is configured twice
2017-03-25 17:52:16 +01:00
Alexander Neumann
946b4f4b86
Add extended options via -o/--option
2017-03-25 17:52:12 +01:00
Alexander Neumann
d0a5e86da1
Add free-form options parsing
2017-03-25 17:45:11 +01:00
Alexander Neumann
2e0b19f63f
location: Move to backend/location
2017-03-25 17:45:11 +01:00
Alexander Neumann
9861f3d435
location: Reformat tests, use sub tests
2017-03-25 17:45:10 +01:00
Alexander Neumann
482a6e9840
local: Add Config struct
2017-03-25 17:45:10 +01:00
Alexander Neumann
1733b24f9d
doc: Add section about how to change the sftp port
...
Closes #895
Related to #785
2017-03-25 09:07:50 +01:00
Alexander Neumann
6f76a6db66
rest: Make backend honor the REST protocol
2017-03-16 21:50:26 +01:00
trbs
f1ba45723c
introduce RWLock for caching obj.Stat()
2017-03-14 23:05:51 +01:00
trbs
b523eef294
Cache size of last ReadAt on S3 for performance
...
Each obj.Stat() call adds another request to the S3 endpoint
for some commands a lot of ReadAt calls are made for the same
object in S3. This patch essentially cuts the number of calls
to S3 in this case in half. Speeding up the progress and lowering
costs to S3.
2017-03-14 14:02:50 +01:00
Pauline Middelink
642cd3bebf
Fix restoring files as non-root user
...
As we cannot reliably detect in advance if we can set ownership, permissions,
timestamps or ext attributes, execute ALL the requested changes before
returning the first error we found.
Report total errors at end of restore and stop printing entire stacktraces
where just the error message is sufficient.
Fixes #655
2017-03-12 16:51:58 +01:00
Alexander Neumann
887e81188f
Merge pull request #877 from middelink/fix-875
...
Fix crash at restoring pre-existing hardlinked files
2017-03-11 10:13:43 +01:00
Pauline Middelink
b56e16acd0
Fix crash at restoring pre-existing hardlinked files
...
Remove target file, ignore non existing file errors.
Small memory saving: Only keep inodes around for files with a link count > 1.
(We will/can never be asked to restore a hardlinked file with one of the
files having a link count of 1.)
Closes #836
2017-03-11 00:06:58 +01:00
Pauline Middelink
b75186533d
Restore does not truncate files it restores
...
Missing O_TRUNC when doing O_CREAT.
Closes #875
2017-03-10 22:46:13 +01:00
Alexander Neumann
8958efba60
Merge pull request #871 from middelink/fix-869
...
Refactor output of `find` to allow for json and normal output.
2017-03-10 20:03:26 +01:00
Pauline Middelink
a9707a5728
Refactor output of `find` to allow for json and normal output.
...
Rather complicated solution becaused I wanted to retain the streaming
character of the output, which means for json I have to manually add
headers and footers per snapshot scanned + a list around the whole
set.
As the json ouput is now partly handcrafted, add proper testing to catch
unintentional changes to the output, making it non-json compliant.
Closes #869
2017-03-10 16:23:58 +01:00
Alexander Neumann
deddedd88f
Merge pull request #872 from middelink/fix-787
...
Suppress (non error) output of `forget` during -q
2017-03-09 21:23:36 +01:00
Pauline Middelink
b38294f236
Suppress (non error) output of `forget` during -q
...
Closes #787
2017-03-09 20:20:17 +01:00
Pauline Middelink
8a05de537f
Refactor `prune` and `rebuild_index`
...
Factor out and reuse `rebuildIndex()` in cmd_rebuild_index and cmd_prune.
Use contexts.
2017-03-08 20:30:52 +01:00
Pauline Middelink
8a92687d9a
Refactor `find` and `ls` commands
...
Implement filtering by using `FindFilteredSnapshots()` to iterate over the snapshots
Refactor cmd_ls' `PrintNode()` into format.go, reuse its pretty printing in both `find`
and `ls` commands.
Use contexts.
2017-03-08 20:29:31 +01:00
Pauline Middelink
3432e7edcd
Refactor `tag` to use `FindFilteredSnapshots()`
2017-03-08 20:28:44 +01:00
Pauline Middelink
3c6c17abcd
Refactor `forget` and `snapshots` command
...
Implement filtering by using `FindFilteredSnapshots()` to iterate over the snapshots
Refactor cmd_snapshots' `PrintSnapshots()` so its pretty printing can be used from
both `forget` and `snapshots`.
Use contexts.
2017-03-08 20:24:58 +01:00
Pauline Middelink
11d237c252
New helper function `FindFilteredSnapshots` to iterate over snapshots
...
This helper function takes a set of filters and/or a list of snapshots
from the commandline. It returns a channel of *Snapshot.
When snapshot ids are given, they are checked for validity and their
corresponding Snapshots returned. The snapshot id "latest" is handled
special to return either the last snapshot (no filters) or the last
snapshot matching the filters.
When no arguments are given, the filters are applied over all available
snapshots and these are returned.
2017-03-08 20:19:12 +01:00
Pauline Middelink
0f7b6ec5ac
Adapt `key` command to context world.
2017-03-08 20:17:30 +01:00
Pauline Middelink
b4526c4e6e
Enable the use of `context` in restic
...
Set up a cancelble context in global options, hook it into the ctrl-C handler
for proper cancel propegation.
Bump up minimal requirement for Go to version 1.7 in documentation
and test-build files.
2017-03-08 20:12:16 +01:00
Pauline Middelink
3eaaa0f286
Correct some typo's in comments.
2017-03-08 20:09:24 +01:00
Pauline Middelink
b1c8071163
Add filtering to `mount` command
2017-03-08 19:59:19 +01:00
Pauline Middelink
3468108d4c
Implement --tag processing to `backup` and `restore` command
...
Add `tags` argument to `FindLatestSnapshot()`
2017-03-08 19:55:58 +01:00
Alexander Neumann
33c8dd4ee5
Merge pull request #867 from middelink/optimize
...
Optimize List() pipeline
2017-03-08 11:50:47 +01:00
Pauline Middelink
7c989ca487
Optimize List() pipeline
2017-03-07 20:58:36 +01:00
Pauline Middelink
e9a2982ecd
Add `--path` for snapshot filtering by path.
...
Add `--group-by-tags` for grouping on host,tags,dirs instead of host,dirs.
Borrow the snapshot printing from cmd_snapshot.
Closes #841
2017-03-07 19:28:42 +01:00
Alexander Neumann
f5a55a81f7
Merge pull request #865 from restic/handle-empty-snapshots
...
Refuse to create empty snapshots
2017-03-07 11:21:48 +01:00
Alexander Neumann
340f2c80a0
Merge pull request #864 from restic/find-case-insensitive
...
find: Add option to ignore case
2017-03-07 11:20:33 +01:00
Alexander Neumann
bb144436c7
Add test for empty snapshot
2017-03-07 11:17:15 +01:00