Commit Graph

50 Commits

Author SHA1 Message Date
Christian Kemper 24b7514fe0 cleaner sharing of s3: and s3:// configuration 2016-02-14 09:45:58 -08:00
Christian Kemper 91dc14c9fc fix Hound warning 2016-02-14 09:27:00 -08:00
Christian Kemper 32c2cafa89 Simplify creation of the Config by moving it to a separate function. Simplify the parsing logic
by sharing the handling of s3: and s3://
2016-02-14 09:18:22 -08:00
Christian Kemper 74608531c7 strip off the trailing slash for the object prefix 2016-02-14 07:16:50 -08:00
Christian Kemper 48f85fbb09 replaced if-else chain with switch 2016-02-14 07:01:14 -08:00
Christian Kemper 535dfaf097 address first round of review comments 2016-02-14 06:40:15 -08:00
Christian Kemper 8f5ff379b7 Introduced a configurable object path prefix for s3 repositories.
Prepends the object path prefix to all s3 paths and allows to have multiple independent
restic backup repositories in a single s3 bucket.

Removed the hardcoded "restic" prefix from s3 paths.

Use "restic" as the default object path prefix for s3 if no other prefix gets specified.
This will retain backward compatibility with existing s3 repository configurations.

Simplified the parse flow to have a single point where we parse the bucket name and the prefix within the bucket.

Added tests for s3 object path prefix and the new default prefix to config_test and location_test.
2016-02-14 06:05:38 -08:00
Alexander Neumann 2701eabe39 Remove ContinuousReader 2016-01-26 22:35:51 +01:00
Alexander Neumann c388101217 s3: Unexport structure 2016-01-26 22:19:44 +01:00
Alexander Neumann d9c87559b5 s3/local backend: Fix error for overwriting files 2016-01-24 21:13:24 +01:00
Alexander Neumann 1547d3b656 Remove Create() everywhere 2016-01-24 20:23:50 +01:00
Alexander Neumann 54f8860612 backends: Add Save() 2016-01-24 16:59:38 +01:00
Alexander Neumann ed172c06e0 backends: Add Save() function 2016-01-24 01:15:35 +01:00
Alexander Neumann adbe9e2e1c backend: Remove GetReader 2016-01-24 01:00:27 +01:00
Alexander Neumann 919b40c6cf Add Stat() method to backend interface 2016-01-23 23:27:58 +01:00
Alexander Neumann f05a32509e Add "Test" prefix to backend test functions 2016-01-23 19:12:02 +01:00
Alexander Neumann 15c8b85a4b Add tests for s3 backend 2016-01-23 18:46:04 +01:00
Alexander Neumann 0a24261afb Add Load() for all existing backends 2016-01-23 14:12:12 +01:00
Alexander Neumann 8b7bf8691d backend: Remove Get()
This is the first commit that removes the (redundant) Get() method of
the backend interface. Get(x, y) is equivalent to GetReader(x, y, 0, 0).
2016-01-23 13:13:05 +01:00
Alexander Neumann c6e1696f07 Fix debug message 2016-01-17 18:48:05 +01:00
Alexander Neumann 289aee9448 Adapt s3 backend to new library 2016-01-17 18:48:05 +01:00
Alexander Neumann 61e66e936f Fix imports 2016-01-17 18:48:05 +01:00
Alexander Neumann 314182e7e0 Add debug, do not create bucket if it already exists 2016-01-17 18:48:05 +01:00
Alexander Neumann 5722ccfcda Fix s3 backend, add more tests 2016-01-17 18:48:05 +01:00
Alexander Neumann a850041cf0 ContReader: Remove debug output 2016-01-17 18:48:05 +01:00
Alexander Neumann e0361b1f9f Add ContinuousReader 2016-01-17 18:48:05 +01:00
Alexander Neumann d79c85af62 Fix s3 tests 2016-01-17 18:48:05 +01:00
Alexander Neumann 407819e5a9 s3: properly integrate minio-go lib 2016-01-17 18:48:05 +01:00
Alexander Neumann 1922a4272c s3: fix usage
Ignore error response for existing bucket, add more debug code.
2016-01-17 18:48:05 +01:00
Alexander Neumann 2b0b44c5ce s3: implement open with config 2016-01-17 18:48:05 +01:00
Alexander Neumann de933a1d48 Rename URI -> Config/Location 2016-01-17 18:48:05 +01:00
Alexander Neumann 566a15285a Add repository location parsing code 2016-01-17 18:48:05 +01:00
Alexander Neumann 5736742c3e s3: Open() creates bucket if it does not exist 2016-01-17 18:46:08 +01:00
Alexander Neumann 248f991ad4 s3: don't remove the bucket on Delete() 2016-01-17 18:46:08 +01:00
Alexander Neumann 55f10eb1c1 Fix s3 test with local minio server instance 2016-01-17 18:46:08 +01:00
Alexander Neumann d0ca118387 Fix usage of the `done` chan 2016-01-17 18:46:08 +01:00
Chris Howey 69a9adc4c3 Use local instance of minio server.
Need to figure out how to have tests automatically start and kill
server.
2016-01-17 18:46:08 +01:00
Chris Howey e2445f4c97 GetPartialObject does not work. 2016-01-17 18:46:08 +01:00
Chris Howey 6d1552af51 Switch s3 library to allow for s3 compatible backends. Fixes #315 2016-01-17 18:46:08 +01:00
Chris Howey 10cd672a92 Use new version of s3 library, Fixes #276 2015-08-26 06:25:05 -05:00
Emil Hessman 52b6cfbbfa backend: fix doc typos 2015-06-28 09:44:06 +02:00
Chris Howey cf0531a2b1 Pass buffer instead of byte slice
s3 library creates internal buffer pass in byte slice
2015-06-14 09:05:20 -05:00
Chris Howey 57ee1db8aa Rename backend type to S3Backend 2015-06-14 08:28:50 -05:00
Chris Howey e963faf846 Place all restic data under a key prefix.
Deleting a restic repository now only deletes restic keys, leaving the
rest of the bucket in tact.
2015-06-14 08:28:50 -05:00
Chris Howey d30a95e8ad Return bucket readers instead of buffering a key on read. 2015-06-14 08:28:50 -05:00
Chris Howey aaae7f33d3 Have number of connections limited by channel
Removes previous limit of 1 connection
2015-06-14 08:28:50 -05:00
Chris Howey 2350419f59 Modify test suite to use s3 test server.
Fix s3 backend to return error on attempting to write to already
existing key.
2015-06-14 08:28:50 -05:00
Chris Howey 4aa75994b0 Add ability to open bucket using S3 object for testing purposes. 2015-06-14 08:28:50 -05:00
Chris Howey bfe221e71c Handle long list responses from s3.
Also change naming of backend struct.
2015-06-14 08:28:50 -05:00
Chris Howey 47186806cf Add Amazon S3 backend. 2015-06-14 08:28:50 -05:00