Alexander Neumann
|
367cc75970
|
chunker: Refactor to take polynomial on creation
|
2015-04-05 22:52:55 +02:00 |
|
Alexander Neumann
|
3cdf3a25b9
|
chunker: Require a random irreducible polynomial
This also implements the necessary polynomial arithmetics in F_2[X].
|
2015-04-05 22:52:43 +02:00 |
|
Alexander Neumann
|
59cfc4bd03
|
Slightly improve performance for chunker
benchcmp output:
benchmark old ns/op new ns/op delta
BenchmarkChunkerWithSHA256 151752169 148072829 -2.42%
BenchmarkChunkerWithMD5 97166433 93357090 -3.92%
BenchmarkChunker 77265146 73642723 -4.69%
benchmark old MB/s new MB/s speedup
BenchmarkChunkerWithSHA256 69.10 70.81 1.02x
BenchmarkChunkerWithMD5 107.92 112.32 1.04x
BenchmarkChunker 135.71 142.39 1.05x
benchmark old allocs new allocs delta
BenchmarkChunkerWithSHA256 15 13 -13.33%
BenchmarkChunkerWithMD5 14 12 -14.29%
BenchmarkChunker 8 6 -25.00%
benchmark old bytes new bytes delta
BenchmarkChunkerWithSHA256 262924 26821 -89.80%
BenchmarkChunkerWithMD5 262774 13554 -94.84%
BenchmarkChunker 262678 13458 -94.88%
|
2015-02-13 20:03:40 +01:00 |
|
Alexander Neumann
|
3bf66d24fa
|
chunker: Add test, only pass hash.Hash to New()
|
2015-02-11 13:12:36 +01:00 |
|
Alexander Neumann
|
3daeee8e2c
|
Make chunker resettable
|
2015-02-09 23:37:33 +01:00 |
|
Alexander Neumann
|
bda33e612c
|
Implement streaming chunker using io.Reader
|
2015-02-09 00:28:54 +01:00 |
|
Alexander Neumann
|
a5c33d80d8
|
Chunker: remove pool, buf and make bufsize an option
|
2015-02-08 19:32:12 +01:00 |
|
Alexander Neumann
|
bdcdcdea7d
|
Optimize chunker
Skip MinSize bytes at the beginning of each chunk. This increases
throughput about 30% percent.
Before:
$ go test -v -bench . ./chunker
=== RUN TestChunker
--- PASS: TestChunker (0.60s)
=== RUN TestChunkerReuse
--- PASS: TestChunkerReuse (2.18s)
PASS
BenchmarkChunker 10 111229642 ns/op 94.27 MB/s
--- BENCH: BenchmarkChunker
chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
ok github.com/restic/restic/chunker 4.120s
After:
$ go test -v -bench . ./chunker
=== RUN TestChunker
--- PASS: TestChunker (0.48s)
=== RUN TestChunkerReuse
--- PASS: TestChunkerReuse (1.75s)
PASS
BenchmarkChunker 20 81468596 ns/op 128.71 MB/s
--- BENCH: BenchmarkChunker
chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
chunker_test.go:168: 6 chunks, average chunk size: 1747626 bytes
ok github.com/restic/restic/chunker 4.061s
|
2015-01-14 16:33:41 +01:00 |
|
Alexander Neumann
|
9dc0bf6378
|
Fix reusing chunker
|
2014-12-03 23:20:45 +01:00 |
|
Alexander Neumann
|
f95788ed90
|
Limit memory usage, add several sync.Pool
|
2014-11-23 21:26:20 +01:00 |
|
Alexander Neumann
|
d594cd89b7
|
Chunker: remove unneeded return
|
2014-11-19 22:56:52 +01:00 |
|
Alexander Neumann
|
83ea81d8c3
|
Fix special case with null byte input
|
2014-10-03 21:44:39 +02:00 |
|
Alexander Neumann
|
878a96e761
|
Fix missing byte
|
2014-09-18 22:34:08 +02:00 |
|
Alexander Neumann
|
1c03bd7f7b
|
Clarify behaviour of Next()
|
2014-09-18 22:22:44 +02:00 |
|
Alexander Neumann
|
8e24a8b811
|
Add more documentation and error handling
|
2014-09-18 22:20:12 +02:00 |
|
Alexander Neumann
|
18131f88bf
|
Add code for content defined chunking (CDC)
|
2014-09-18 21:10:20 +02:00 |
|