Commit Graph

2341 Commits

Author SHA1 Message Date
Alexander Neumann 1e0e6ee573 s3: Use low-level API with a Range header for Load
benchmark                                         old ns/op      new ns/op      delta
    BenchmarkBackendMinio/LoadFile-4                  9213315        11001787       +19.41%
    BenchmarkBackendMinio/LoadPartialFile-4           4176619        3479707        -16.69%
    BenchmarkBackendMinio/LoadPartialFileOffset-4     4391521        3139214        -28.52%
    BenchmarkBackendS3/LoadFile-4                     2886070905     2505907501     -13.17%
    BenchmarkBackendS3/LoadPartialFile-4              762702722      735694398      -3.54%
    BenchmarkBackendS3/LoadPartialFileOffset-4        789724328      1108989142     +40.43%

    benchmark                                         old MB/s     new MB/s     speedup
    BenchmarkBackendMinio/LoadFile-4                  1821.21      1525.15      0.84x
    BenchmarkBackendMinio/LoadPartialFile-4           1004.49      1205.67      1.20x
    BenchmarkBackendMinio/LoadPartialFileOffset-4     955.34       1336.45      1.40x
    BenchmarkBackendS3/LoadFile-4                     5.81         6.70         1.15x
    BenchmarkBackendS3/LoadPartialFile-4              5.50         5.70         1.04x
    BenchmarkBackendS3/LoadPartialFileOffset-4        5.31         3.78         0.71x

    benchmark                                         old allocs     new allocs     delta
    BenchmarkBackendMinio/LoadFile-4                  406            204            -49.75%
    BenchmarkBackendMinio/LoadPartialFile-4           225            206            -8.44%
    BenchmarkBackendMinio/LoadPartialFileOffset-4     227            207            -8.81%
    BenchmarkBackendS3/LoadFile-4                     600            388            -35.33%
    BenchmarkBackendS3/LoadPartialFile-4              416            302            -27.40%
    BenchmarkBackendS3/LoadPartialFileOffset-4        417            303            -27.34%

    benchmark                                         old bytes     new bytes     delta
    BenchmarkBackendMinio/LoadFile-4                  29475         13904         -52.83%
    BenchmarkBackendMinio/LoadPartialFile-4           4218838       13958         -99.67%
    BenchmarkBackendMinio/LoadPartialFileOffset-4     4219175       14332         -99.66%
    BenchmarkBackendS3/LoadFile-4                     114152        97424         -14.65%
    BenchmarkBackendS3/LoadPartialFile-4              4265416       56212         -98.68%
    BenchmarkBackendS3/LoadPartialFileOffset-4        4266520       56308         -98.68%
2017-05-14 20:19:12 +02:00
Alexander Neumann be0e53c07b tests: Add test for backend Save() from file 2017-05-14 19:55:32 +02:00
Alexander Neumann 9dd58196e3 Merge pull request #961 from restic/add-backend-benchmarks
Add backend benchmarks
2017-05-14 19:49:33 +02:00
Alexander Neumann 66b4999765 tests: Remove code generation, use reflection
This simplifies the code.
2017-05-14 12:59:07 +02:00
Alexander Neumann 403e201e1a tests: Improve robustness of config tests 2017-05-14 12:50:20 +02:00
Alexander Neumann 77a55fbe5c tests: Add documentation 2017-05-14 11:48:30 +02:00
Alexander Neumann 90c1608d88 sftp: Add Benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann 7a51640262 rest: Add benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann 13946e7db7 mem: Add benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann e009c002ba local: Add benchmark 2017-05-14 11:48:30 +02:00
Alexander Neumann d24e0cc6cc s3: Add benchmarks 2017-05-14 11:48:30 +02:00
Alexander Neumann 5b8131e2d3 tests: Add benchmark for Save 2017-05-14 11:48:30 +02:00
Alexander Neumann 8fc25cc567 tests: Add benchmarks for partial file (+offset) 2017-05-14 11:48:30 +02:00
Alexander Neumann 77ebb95d3d tests: Add BenchmarkLoadFile 2017-05-14 11:48:30 +02:00
Alexander Neumann f142b1c22f tests: Add benchmarks to test suite 2017-05-14 11:48:30 +02:00
Alexander Neumann 1c9159d6a0 Merge pull request #960 from restic/travis-run-tests-against-s3
Enable CI tests against AWS S3
2017-05-14 10:26:16 +02:00
Alexander Neumann 82e15dc6dc Allow skipping the test against s3
If RESTIC_TEST_S3_REPOSITORY is not available it means we're probably
running on Travis for a third-party PR, so the access credentials for S3
are not exposed.
2017-05-14 00:33:19 +02:00
Alexander Neumann ee68f9298b tests: Ensure that backend tests cannot be skipped on Travis 2017-05-14 00:33:15 +02:00
Alexander Neumann 5c6ec78789 s3: Remove log line 2017-05-13 19:58:25 +02:00
Alexander Neumann 4ac0d3ad40 backend tests load: Use reader with Size() method 2017-05-13 19:56:11 +02:00
Harshavardhana c7209ef231 api: Failed call retry with region only when http.StatusBadRequest.
Fixes https://github.com/minio/minio-go/issues/677
2017-05-13 19:53:24 +02:00
Alexander Neumann 0249c16b04 Revert "minio: Apply fix"
This reverts commit f43d34899d.
2017-05-13 19:41:55 +02:00
Alexander Neumann f43d34899d minio: Apply fix
https://github.com/minio/minio-go/issues/677
2017-05-13 13:18:42 +02:00
Alexander Neumann 19daefd04e Merge pull request #938 from restic/rework-backend-tests
WIP: rework backend integration tests
2017-05-12 22:09:57 +02:00
Alexander Neumann ec5e984ed9 Increase waiting time for network services 2017-05-12 21:45:18 +02:00
Alexander Neumann 88de3cfecc rest: Wait until REST server is started 2017-05-12 21:33:34 +02:00
Alexander Neumann a654f41ddb Add docstring for package test 2017-05-12 21:19:32 +02:00
Alexander Neumann fbf2462325 Rename backend test functions 2017-05-12 21:06:49 +02:00
Alexander Neumann 85e4831198 Remove stuff from run_integration_tests.go 2017-05-12 21:04:04 +02:00
Alexander Neumann 091dc29f72 doc: Fix s3 tutorial 2017-05-12 21:04:04 +02:00
Alexander Neumann 59d1986660 s3: Use random prefix for tests 2017-05-12 21:04:04 +02:00
Alexander Neumann db4fa48f66 rest: Add context 2017-05-12 21:04:04 +02:00
Alexander Neumann f2b83ece10 s3: Ignore errors for terminated minio server 2017-05-12 21:04:04 +02:00
Alexander Neumann 860b52273e s3: Add tests for AWS 2017-05-12 21:04:04 +02:00
Alexander Neumann bafceb56fb s3: Convert to test suite, run Minio automatically 2017-05-12 21:04:04 +02:00
Alexander Neumann 9271b3662a sftp: ignore error 2017-05-12 21:04:04 +02:00
Alexander Neumann 554013ca9f rest: Convert to test suite 2017-05-12 21:04:04 +02:00
Alexander Neumann 3250fdc2ca mem: Convert to test suite 2017-05-12 21:04:04 +02:00
Alexander Neumann 3678d34b16 sftp: Convert to test suite 2017-05-12 21:04:04 +02:00
Alexander Neumann 16276853a1 local: Convert to test suite 2017-05-12 21:04:04 +02:00
Alexander Neumann f10c24e404 Improve test helpers 2017-05-12 21:04:04 +02:00
Alexander Neumann 6f5fd72738 Implement test suite for backend tests 2017-05-12 21:03:55 +02:00
Alexander Neumann a963052d64 Add custom HTTP transport 2017-05-11 21:51:30 +02:00
Alexander Neumann 898613e14f debug: Add RoundTripper 2017-05-11 21:51:30 +02:00
Alexander Neumann b84e63d503 backend.LoadAll: Check errors from io.Copy() 2017-05-11 21:51:30 +02:00
Alexander Neumann 63870d2830 Check error for Close() 2017-05-11 21:51:30 +02:00
Alexander Neumann 54e46f5984 Add MinimalData option for backend tests 2017-05-11 21:51:30 +02:00
Alexander Neumann 0096eca7fe Install log buffer for global logs
This hides logged message from the net/http library and only shows them
in case an error occurs.
2017-05-11 21:51:30 +02:00
Alexander Neumann c5ae5524ff Remove codecov.yml 2017-05-10 20:52:06 +02:00
Alexander Neumann fbf5a8123b Merge pull request #952 from restic/remove-temp-files
Unlink tempfiles before using them
2017-05-10 20:25:07 +02:00