rdelaage
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,566 Commits 7 Branches 44 Tags 75 MiB
Commit Graph

821 Commits

Author SHA1 Message Date
Alexander Neumann 40a61b82ce Merge pull request #978 from restic/add-backblaze-backend 
Add Backblaze B2 backend
 2017-06-03 14:54:04 +02:00
Alexander Neumann fec89f95fb Improve swift backend 2017-06-03 14:28:18 +02:00
Bartłomiej Święcki 5681d41f76 Implement OpenStack swift backend 
This commit implements support for OpenStack swift
storage server, tested on OVH public cloud storage.

Special thanks to jayme-github <tuxnet@gmail.com>
who helped with the implementation.
 2017-06-03 14:26:29 +02:00
Alexander Neumann 122462b9b1 Add Backblaze B2 backend 
This is based on prior work by Joe Turgeon <arithmetric@gmail.com>
@arithmetric.
 2017-06-03 14:24:59 +02:00
Alexander Neumann aa0f874c8d s3: Simplify IsNotExist() 2017-05-31 21:23:01 +02:00
Alexander Neumann 5c59484d2b s3: Return only basename in List() 2017-05-31 21:22:55 +02:00
Alexander Neumann 04b262d8f1 Allow many idle connections per host 
Closes #985
 2017-05-31 19:39:19 +02:00
Alexander Neumann f3f6924b61 backend/test: Loose requirement about early error 2017-05-28 13:06:27 +02:00
Alexander Neumann c5244abad9 rest: Improve error messages 2017-05-28 12:33:47 +02:00
Alexander Neumann 1f5954e2c1 layout: Test DefaultLayout for empty path prefix 2017-05-28 12:33:47 +02:00
Alexander Neumann e046a2a6da sftp: Use path instead of filepath 2017-05-28 12:33:47 +02:00
Alexander Neumann 8395b53400 backend/test: Reduce verbosity in logs 2017-05-28 12:33:47 +02:00
Alexander Neumann 24ec14738d backend/test: Skip offset == length test 2017-05-28 12:33:47 +02:00
Alexander Neumann 79477fdfe4 backend/test: Randomize test suite 2017-05-28 12:33:47 +02:00
Alexander Neumann 7ec0543af3 testing: Add id to error message in panic 2017-05-28 10:17:04 +02:00
Alexander Neumann 5247de552a Remove regular status printing for non terminals 2017-05-25 17:03:48 +02:00
Alexander Neumann d1cc87ba28 Merge pull request #966 from restic/unify-repo-layout 
WIP: Unify repository layout
 2017-05-22 21:15:38 +02:00
Alexander Neumann 69e114c5c4 Merge pull request #968 from mcastilho/patch-2 
Fixing CPUProfile
 2017-05-17 21:13:42 +02:00
Marcio Castilho 84f1037b8f Update global_debug.go 
The variable was wrong and the CPU profile was never being created.
 2017-05-17 11:07:33 -04:00
Pauline Middelink 120af801cf Fix golint warnings 
(except the exported fields/functions without comments)
 2017-05-17 01:39:39 +02:00
Pauline Middelink f3d09ce7c8 Fix vet warnings 2017-05-17 01:34:33 +02:00
Pauline Middelink 2b9323529f Fix gofmt -s warnings 2017-05-17 01:28:39 +02:00
Pauline Middelink 971ecee171 Fix ineffassign mistakes 2017-05-17 01:25:52 +02:00
Alexander Neumann 8b461a7456 Make TestFlags less verbose 2017-05-16 20:49:18 +02:00
Alexander Neumann 2fa1238b8a Fix filenames for layout tests 2017-05-16 20:45:17 +02:00
Alexander Neumann fa41183a53 s3: Add s3.layout option and layout auto detection 2017-05-15 23:37:02 +02:00
Alexander Neumann 959aa0f595 Simplify layout detection 2017-05-15 23:36:23 +02:00
Alexander Neumann 069752cb42 Make layout default Dirname() consistent 
Always return a trailing slash now.
 2017-05-15 23:35:52 +02:00
Alexander Neumann 0c537837d9 Rename testdata files 2017-05-15 22:39:50 +02:00
Alexander Neumann f19852a738 Rename CloudLayout -> RESTLayout 
The REST backend uses a special layout without subdirs below data/. This
layout is just used there and nowhere else, and our REST server
implementation uses the default layout for the on disk storage. So we
remove the REST layout from the auto detection code.
 2017-05-15 22:05:43 +02:00
Alexander Neumann 61cade6222 Rename layout s3 -> s3legacy 2017-05-15 21:59:28 +02:00
Alexander Neumann 13393c76dc Merge pull request #957 from middelink/fix-953 
Change backup policy to be inclusive
 2017-05-15 17:19:04 +02:00
Pauline Middelink 7fffd408af Do not use singleton counter. 
Revert change for running the numbered tests as subtests.
 2017-05-15 08:53:59 +02:00
Alexander Neumann 250a45ab15 s3: Move interfaces to function 2017-05-14 20:36:26 +02:00
Alexander Neumann 3b44b87137 s3: Remove file after usage in test 2017-05-14 20:34:22 +02:00
Alexander Neumann 26c16b9fd3 s3: Correctly return token in Load 2017-05-14 20:19:12 +02:00
Alexander Neumann 0bd40bae6e s3: Prevent net/http client from closing the reader 2017-05-14 20:19:12 +02:00
Alexander Neumann 246ccf09b9 s3: add getRemainingSize 2017-05-14 20:19:12 +02:00
Alexander Neumann 9452f416bf s3: Use low level API for saving files 
benchmark                        old ns/op       new ns/op       delta
    BenchmarkBackendMinio/Save-4     184482294       40663344        -77.96%
    BenchmarkBackendS3/Save-4        35030825568     54475455819     +55.51%

    benchmark                        old MB/s     new MB/s     speedup
    BenchmarkBackendMinio/Save-4     90.95        412.64       4.54x
    BenchmarkBackendS3/Save-4        0.48         0.31         0.65x

    benchmark                        old allocs     new allocs     delta
    BenchmarkBackendMinio/Save-4     631            560            -11.25%
    BenchmarkBackendS3/Save-4        646            584            -9.60%

    benchmark                        old bytes     new bytes     delta
    BenchmarkBackendMinio/Save-4     66818060      50735         -99.92%
    BenchmarkBackendS3/Save-4        66834000      73024         -99.89%
 2017-05-14 20:19:12 +02:00
Alexander Neumann 1e0e6ee573 s3: Use low-level API with a Range header for Load 
benchmark                                         old ns/op      new ns/op      delta
    BenchmarkBackendMinio/LoadFile-4                  9213315        11001787       +19.41%
    BenchmarkBackendMinio/LoadPartialFile-4           4176619        3479707        -16.69%
    BenchmarkBackendMinio/LoadPartialFileOffset-4     4391521        3139214        -28.52%
    BenchmarkBackendS3/LoadFile-4                     2886070905     2505907501     -13.17%
    BenchmarkBackendS3/LoadPartialFile-4              762702722      735694398      -3.54%
    BenchmarkBackendS3/LoadPartialFileOffset-4        789724328      1108989142     +40.43%

    benchmark                                         old MB/s     new MB/s     speedup
    BenchmarkBackendMinio/LoadFile-4                  1821.21      1525.15      0.84x
    BenchmarkBackendMinio/LoadPartialFile-4           1004.49      1205.67      1.20x
    BenchmarkBackendMinio/LoadPartialFileOffset-4     955.34       1336.45      1.40x
    BenchmarkBackendS3/LoadFile-4                     5.81         6.70         1.15x
    BenchmarkBackendS3/LoadPartialFile-4              5.50         5.70         1.04x
    BenchmarkBackendS3/LoadPartialFileOffset-4        5.31         3.78         0.71x

    benchmark                                         old allocs     new allocs     delta
    BenchmarkBackendMinio/LoadFile-4                  406            204            -49.75%
    BenchmarkBackendMinio/LoadPartialFile-4           225            206            -8.44%
    BenchmarkBackendMinio/LoadPartialFileOffset-4     227            207            -8.81%
    BenchmarkBackendS3/LoadFile-4                     600            388            -35.33%
    BenchmarkBackendS3/LoadPartialFile-4              416            302            -27.40%
    BenchmarkBackendS3/LoadPartialFileOffset-4        417            303            -27.34%

    benchmark                                         old bytes     new bytes     delta
    BenchmarkBackendMinio/LoadFile-4                  29475         13904         -52.83%
    BenchmarkBackendMinio/LoadPartialFile-4           4218838       13958         -99.67%
    BenchmarkBackendMinio/LoadPartialFileOffset-4     4219175       14332         -99.66%
    BenchmarkBackendS3/LoadFile-4                     114152        97424         -14.65%
    BenchmarkBackendS3/LoadPartialFile-4              4265416       56212         -98.68%
    BenchmarkBackendS3/LoadPartialFileOffset-4        4266520       56308         -98.68%
 2017-05-14 20:19:12 +02:00
Alexander Neumann be0e53c07b tests: Add test for backend Save() from file 2017-05-14 19:55:32 +02:00
Alexander Neumann 66b4999765 tests: Remove code generation, use reflection 
This simplifies the code.
 2017-05-14 12:59:07 +02:00
Alexander Neumann 403e201e1a tests: Improve robustness of config tests 2017-05-14 12:50:20 +02:00
Alexander Neumann 77a55fbe5c tests: Add documentation 2017-05-14 11:48:30 +02:00
Alexander Neumann 90c1608d88 sftp: Add Benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann 7a51640262 rest: Add benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann 13946e7db7 mem: Add benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann e009c002ba local: Add benchmark 2017-05-14 11:48:30 +02:00
Alexander Neumann d24e0cc6cc s3: Add benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann 5b8131e2d3 tests: Add benchmark for Save 2017-05-14 11:48:30 +02:00
Alexander Neumann 8fc25cc567 tests: Add benchmarks for partial file (+offset) 2017-05-14 11:48:30 +02:00
Alexander Neumann 77ebb95d3d tests: Add BenchmarkLoadFile 2017-05-14 11:48:30 +02:00
Alexander Neumann f142b1c22f tests: Add benchmarks to test suite 2017-05-14 11:48:30 +02:00
Alexander Neumann ee68f9298b tests: Ensure that backend tests cannot be skipped on Travis 2017-05-14 00:33:15 +02:00
Alexander Neumann 5c6ec78789 s3: Remove log line 2017-05-13 19:58:25 +02:00
Alexander Neumann 4ac0d3ad40 backend tests load: Use reader with Size() method 2017-05-13 19:56:11 +02:00
Alexander Neumann ec5e984ed9 Increase waiting time for network services 2017-05-12 21:45:18 +02:00
Alexander Neumann 88de3cfecc rest: Wait until REST server is started 2017-05-12 21:33:34 +02:00
Alexander Neumann a654f41ddb Add docstring for package test 2017-05-12 21:19:32 +02:00
Alexander Neumann fbf2462325 Rename backend test functions 2017-05-12 21:06:49 +02:00
Alexander Neumann 59d1986660 s3: Use random prefix for tests 2017-05-12 21:04:04 +02:00
Alexander Neumann db4fa48f66 rest: Add context 2017-05-12 21:04:04 +02:00
Alexander Neumann f2b83ece10 s3: Ignore errors for terminated minio server 2017-05-12 21:04:04 +02:00
Alexander Neumann 860b52273e s3: Add tests for AWS 2017-05-12 21:04:04 +02:00
Alexander Neumann bafceb56fb s3: Convert to test suite, run Minio automatically 2017-05-12 21:04:04 +02:00
Alexander Neumann 9271b3662a sftp: ignore error 2017-05-12 21:04:04 +02:00
Alexander Neumann 554013ca9f rest: Convert to test suite 2017-05-12 21:04:04 +02:00
Alexander Neumann 3250fdc2ca mem: Convert to test suite 2017-05-12 21:04:04 +02:00
Alexander Neumann 3678d34b16 sftp: Convert to test suite 2017-05-12 21:04:04 +02:00
Alexander Neumann 16276853a1 local: Convert to test suite 2017-05-12 21:04:04 +02:00
Alexander Neumann f10c24e404 Improve test helpers 2017-05-12 21:04:04 +02:00
Alexander Neumann 6f5fd72738 Implement test suite for backend tests 2017-05-12 21:03:55 +02:00
Pauline Middelink 103a491ac0 Make houndci-bot happy. 2017-05-11 22:38:12 +02:00
Pauline Middelink 929f90344e Change backup policy to be inclusive, meaning all given policies 
are evaluated for each snapshot, thereby making sure that each
keep-* is able to retain its most recent snapshot. Thereby insuring
that weeklies keep Sundays around and monthlies keep the last day of
the month around.

Added testcase to make sure when multiple --keep-tags are given,
ALL of them need to match.
 2017-05-11 22:34:50 +02:00
Alexander Neumann a963052d64 Add custom HTTP transport 2017-05-11 21:51:30 +02:00
Alexander Neumann 898613e14f debug: Add RoundTripper 2017-05-11 21:51:30 +02:00
Alexander Neumann b84e63d503 backend.LoadAll: Check errors from io.Copy() 2017-05-11 21:51:30 +02:00
Alexander Neumann 63870d2830 Check error for Close() 2017-05-11 21:51:30 +02:00
Alexander Neumann 54e46f5984 Add MinimalData option for backend tests 2017-05-11 21:51:30 +02:00
Alexander Neumann 0096eca7fe Install log buffer for global logs 
This hides logged message from the net/http library and only shows them
in case an error occurs.
 2017-05-11 21:51:30 +02:00
Pauline Middelink ce3acbd30a Remove unused SnapshotFilter and FilterSnapshots(). 2017-05-11 21:07:58 +02:00
Alexander Neumann edbd6ad584 Add fs.TempFile and fs.RemoveIfExists 2017-05-10 19:48:22 +02:00
Alexander Neumann bcfe7afbfd Merge pull request #945 from middelink/fix-942 
Add bash autocompletion generation
 2017-05-07 10:30:10 +02:00
Alexander Neumann 46e6e8984d Merge pull request #941 from restic/improve-mkdir 
local/sftp: Auto create directories if needed
 2017-05-07 10:26:39 +02:00
Pauline Middelink 2c42629c51 Add bash autocompletion generation 
Fix #942
 2017-05-06 12:39:23 +02:00
Pauline Middelink f93e5a39e5 Small textual changes to apply, to reflect actual meaning. 
Added tests for the ExpirePolicy operations.
 2017-05-04 16:39:41 +02:00
Pauline Middelink 02eafe94b3 Use global context in cmd_forget, like all other cmds 2017-05-04 16:35:35 +02:00
Alexander Neumann 7d914768bb local/sftp: Auto create directories if needed 
As reported in #940
 2017-05-03 21:19:22 +02:00
Alexander Neumann 548d4eed95 Correct backend test for len = 0 2017-04-26 20:47:15 +02:00
Alexander Neumann ce28584dda Don't sort flags for `forget` command 2017-04-21 19:25:21 +02:00
Alexander Neumann d0db1bf9b3 Test if a config file on open 2017-04-19 18:56:01 +02:00
Alexander Neumann e9d939ff2d Update code for pkg/xattr v0.2.0 2017-04-18 21:39:55 +02:00
Alexander Neumann 28968caf33 s3: Correct prefix for layout 2017-04-17 20:06:55 +02:00
Alexander Neumann 16fd1c2352 s3: Correct layout handle url/path 2017-04-17 20:06:55 +02:00
Alexander Neumann b942f61272 s3: Reduce connection limit, prevents timeouts 2017-04-17 20:06:55 +02:00
Alexander Neumann 69dd1d2544 backup: Allow specifying multiple exclude files 2017-04-16 20:52:41 +02:00
Alexander Neumann dd65ac56ef filter: Ignore empty patterns 2017-04-16 20:49:20 +02:00
Alexander Neumann c796d84fca Ignore empty lines in excludes file 
Closes #915
 2017-04-16 20:46:52 +02:00
Alexander Neumann 525db875b0 Merge pull request #898 from restic/prepare-cloud-backends 
Prepare more cloud backends, add backend layouts
 2017-04-15 11:30:25 +02:00
Alexander Neumann 5bd95b3ce1 Implement MkdirAll() for Windows 
Closes #735
 2017-04-15 10:56:17 +02:00
Alexander Neumann be06983c80 options: Fix sorting (and test) 2017-04-14 00:45:54 +02:00
Alexander Neumann a634c22ae0 Add hidden 'options' command to list all opts 2017-04-14 00:00:03 +02:00
Alexander Neumann 859ee23d2e options: Register local and sftp backends 2017-04-14 00:00:03 +02:00
Alexander Neumann b7671dafc8 options: Allow registering 2017-04-14 00:00:02 +02:00
Alexander Neumann 541484d142 s3: Use Layout 2017-04-14 00:00:02 +02:00
Alexander Neumann f531ca3b48 layout: Fix corner cases 2017-04-14 00:00:02 +02:00
Alexander Neumann 0da7264e75 rest: Convert to Layout 2017-04-14 00:00:02 +02:00
Alexander Neumann 7f3bcdb4cc layout: prepare use for REST backend 2017-04-14 00:00:02 +02:00
Alexander Neumann 5eaa51eeff Remove unused assignments 2017-04-14 00:00:02 +02:00
Alexander Neumann 7b64b890d7 Simplify code 2017-04-14 00:00:02 +02:00
Alexander Neumann ccc201ea5f remove unused code 2017-04-14 00:00:02 +02:00
Alexander Neumann e6578857cf sftp/local: Fix listing files 2017-04-14 00:00:02 +02:00
Alexander Neumann 320c22f1f5 backend/layout: Add Basedir() 2017-04-14 00:00:02 +02:00
Alexander Neumann e2af5890f3 backend: Add test for listing files with layouts 2017-04-14 00:00:02 +02:00
Alexander Neumann 36b1c0898c sftp: Add OS X sftp-server path 2017-04-13 23:59:59 +02:00
Alexander Neumann c723cdf808 Check allowed devices per path 
Closes #645
Closes #702
 2017-04-13 21:03:05 +02:00
Alexander Neumann c2ee0d9c84 sftp: Skip tests if server binary is not available 2017-04-10 22:51:00 +02:00
Alexander Neumann e8780f1ec6 sftp: Add layout tests 2017-04-10 22:42:33 +02:00
Alexander Neumann 783fd73ea1 local: Rename local_layout_test 2017-04-10 22:42:33 +02:00
Alexander Neumann 74eb293733 sftp: Remove legacy filename/dirname methods 2017-04-10 22:42:33 +02:00
Alexander Neumann 42ea4d257b sftp: first step of conversion to Layout 2017-04-10 22:42:33 +02:00
Alexander Neumann a849edf19a local: remove double Close() 2017-04-10 22:42:33 +02:00
Alexander Neumann 0cbd59856c layout: Add IsNotExist 2017-04-10 22:42:33 +02:00
Alexander Neumann 698ba57597 backend/tests: Print error stacktrace if available 2017-04-10 22:42:33 +02:00
Alexander Neumann 27ce6a85e9 sftp: Rework Open/Create 2017-04-10 22:42:33 +02:00
Alexander Neumann ae290ab374 sftp: Rename Dir -> Path 2017-04-10 22:42:33 +02:00
Alexander Neumann ab602c9d14 sftp: Add Layout 2017-04-10 22:40:24 +02:00
Alexander Neumann 2e53af1b75 sftp: Rename Open/Create 2017-04-10 22:39:13 +02:00
Alexander Neumann a725e065d9 Merge pull request #903 from restic/fix-hardlinks 
Add new field DeviceID and tests
 2017-04-07 21:05:20 +02:00
Alexander Neumann db7e23b423 Skip /dev/null on darwin 2017-04-07 20:37:20 +02:00
Alexander Neumann 10a395ca33 Make tests runnable on os x/darwin 2017-04-06 20:36:09 +02:00
Alexander Neumann 522c7ade91 Add test for double defined flags 2017-04-06 19:44:53 +02:00
Alexander Neumann a3d6099892 Remove short option conflict 2017-04-06 19:14:38 +02:00
Alexander Neumann 280028290e Disable tests on darwin 2017-04-05 21:35:29 +02:00
Alexander Neumann c195139d31 Only run tests on unix 2017-04-05 20:51:26 +02:00
Alexander Neumann 6f1b03415c Fix hardlinks 2017-04-05 20:45:24 +02:00
Alexander Neumann ffcb015581 Add new field DeviceID and tests 2017-04-05 20:42:15 +02:00
jgfrm 6674b2a70c Moved assignment of device id outside the switch 2017-04-05 11:33:08 +02:00
jgfrm c855d6bb9a storage of device id of files 2017-04-04 21:28:25 +02:00
Alexander Neumann 1086528ab7 sftp: Fix errors import 2017-04-03 21:42:41 +02:00
Alexander Neumann c26dd6b76f sftp: Integrate command 2017-04-03 21:05:42 +02:00
Alexander Neumann d3b6f75848 sftp: Add SplitShellArgs 2017-04-03 08:57:33 +02:00
Alexander Neumann d1efdcd78e Add integration test for layouts 2017-04-02 20:35:17 +02:00
Alexander Neumann 95ab5adda1 local: Expose layout as extended option 2017-04-02 20:29:00 +02:00
Alexander Neumann c5eb36fe9d layout: improve error message for ParseLayout 2017-04-02 20:28:42 +02:00
Alexander Neumann e3e3a8a695 local: Add layout tests 2017-04-02 20:01:40 +02:00
Alexander Neumann 24ebf95f33 local: Automatically detect layout 2017-04-02 20:01:40 +02:00
Alexander Neumann 54465c92cc layout: Allow passing in a default layout 2017-04-02 20:01:36 +02:00
Alexander Neumann f7c4b3a922 Fix layout detection 2017-04-02 19:18:03 +02:00
Alexander Neumann 50dfa64a54 Add layout name parser 2017-04-02 17:57:28 +02:00