Commit Graph

25 Commits

Author SHA1 Message Date
Alexander Neumann fc5439a37a cache: Ensure failed downloads are retried
This fixes #1833, which consists of two different bugs:

 * The `defer` in `cacheFile()` may remove a channel from the
   `inProgress` map although it is not responsible for downloading the
   file

 * If the download fails, goroutines waiting for the file to be cached
   assumed that the file was there, there was no way to signal the
   error.
2018-06-09 17:50:56 +02:00
Gaige Lama a63989afcd
Ran gofmt on backend.go
I ran gofmt on backend.go, this appears to valid edit.
2018-05-11 11:07:16 -04:00
Alexander Neumann f928aeec34 Add 'cache' command 2018-05-01 16:27:44 +02:00
Alexander Neumann 99f7fd74e3 backend: Improve Save()
As mentioned in issue [#1560](https://github.com/restic/restic/pull/1560#issuecomment-364689346)
this changes the signature for `backend.Save()`. It now takes a
parameter of interface type `RewindReader`, so that the backend
implementations or our `RetryBackend` middleware can reset the reader to
the beginning and then retry an upload operation.

The `RewindReader` interface also provides a `Length()` method, which is
used in the backend to get the size of the data to be saved. This
removes several ugly hacks we had to do to pull the size back out of the
`io.Reader` passed to `Save()` before. In the `s3` and `rest` backend
this is actively used.
2018-03-03 15:49:44 +01:00
Igor Fedorenko d58ae43317 Reworked Backend.Load API to retry errors during ongoing download
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
2018-02-16 21:12:14 -05:00
Alexander Neumann 909d9273cc Close backend files in case of errors 2018-01-25 21:05:57 +01:00
Edward Betts b5c23695c9
Correct a spelling mistake. 2018-01-24 12:09:41 +00:00
Alexander Neumann ea593fca1b cache: Correctly return dir for Windows/darwin 2017-11-20 06:11:18 +01:00
Alexander Neumann fe1f151ae1 cache: Return error during default dir detection 2017-11-20 06:10:42 +01:00
Alexander Neumann f4bab789b8 cache: Simplify cache dir creation 2017-11-24 20:53:26 +01:00
Alexander Neumann fa893ee477 cache: Add detection code for old cache dirs 2017-11-24 20:53:26 +01:00
George Armhold bcdebfb84e small cleanup:
- be explicit when discarding returned errors from .Close(), etc.
- remove named return values from funcs when naked return not used
- fix some "err" shadowing when redeclaration not needed
2017-10-25 12:03:55 -04:00
Alexander Neumann 8e2ef3f38b cache: Store first, then store in the backend
Store the file in the backend, then rewind the reader and store it
in the cache afterwards.
2017-10-17 22:01:20 +02:00
Alexander Neumann cebee0b8fa cache: Refuse to cache truncated files 2017-10-05 20:40:02 +02:00
Alexander Neumann d81adcfaa5 cache: Add file name to error message 2017-10-05 19:30:56 +02:00
Michael Pratt 2133869127 cache: OS-specific cache directories
Windows, and to a lesser extent OS X, don't conform to XDG and have
their own preferred locations for caches.

On Windows, use %LOCALAPPDATA%/restic (i.e., ~/AppData/Local/restic). I
can't find authoritative documentation from Microsoft recommending
specifically which of %APPDATA%, %LOCALAPPDATA%, and %TEMP% should be
used for caches, but %LOCALAPPDATA% is where browsers store their
caches, so it seems like a good fit.

On OS X, use ~/Library/Caches/restic, which is recommended by the Apple
documentation. They do suggest using the application "bundle identifier"
as the base folder name, but restic doesn't have one, so I just used
"restic".
2017-09-27 21:16:22 -07:00
Alexander Neumann 0bb2a8e0d0 cache: Synchronize downloading
This commit adds code to synchronize downloading files to the cache.
Before, requests that came in for files currently downloading would fail
because the file was not completed in the cache. Now, the code waits
until the download is completed.

Closes #1278
2017-09-25 15:58:20 +02:00
Alexander Neumann e262f35d0a cache: Auto-remove invalid files 2017-09-24 23:11:47 +02:00
Alexander Neumann 240c4cf2fd cache: In case of an error, fall back backend 2017-09-24 23:11:23 +02:00
Alexander Neumann e1dfaf5d87 cache: Allow proactive caching of tree packs
This commit adds a function to the cache which can decide to proactively
load the complete pack file and store it in the cache. This is helpful
for pack files containing only tree blobs, as it is likely that the same
file is accessed again in the future.
2017-09-24 23:11:23 +02:00
Alexander Neumann 5436154f0d cache: Add PerformReadahead 2017-09-24 23:11:23 +02:00
Alexander Neumann 809e218d20 cache: Improve debug logs 2017-09-24 23:11:23 +02:00
Alexander Neumann 610b676444 Automatically exclude current restic cache 2017-09-24 21:54:53 +02:00
Alexander Neumann 58699e3c90 Write CACHEDIR.TAG to cache base directory 2017-09-24 21:54:53 +02:00
Alexander Neumann 9be24a1c9f Add cache
This commits adds rudimentary support for a cache directory, enabled by
default. The cache directory is created if it does not exist. The cache
is used if there's anything in it, newly created snapshot and index
files are written to the cache automatically.
2017-09-24 21:54:53 +02:00