Commit Graph

59 Commits

Author SHA1 Message Date
Michael Eischer 95a1bb4261 restic: Rework error handling of FindFilteredSnapshots and handle snapshotIDs
FindFilteredSnapshots no longer prints errors during snapshot loading on
stderr, but instead passes the error to the callback to allow the caller
to decide on what to do.

In addition, it moves the logic to handle an explicit snapshot list from
the main package to restic.
2022-10-15 13:31:26 +02:00
Michael Eischer dd7cd5b9b3 fuse: remove unused context parameter 2022-08-26 20:48:48 +02:00
Michael Eischer a0c1ae9f90 mount: Correctly return context.Canceled for interrupted syscalls
bazil/fuse expects us to return context.Canceled to signal that a
syscall was successfully interrupted. Returning a wrapped version of
that error however causes the fuse library to signal an EIO (input/output
error). Thus unwrap context.Canceled errors before returning them.
2022-08-26 20:48:48 +02:00
MichaelEischer 7e96a5af62
Merge pull request #3872 from MichaelEischer/fuse-fix
mount: Only remember successful snapshot refreshes
2022-08-19 19:21:29 +02:00
Michael Eischer 522406b4f0 mount: Only remember successful snapshot refreshes
If the context provided by the fuse library is canceled before the index
was loaded this could lead to missing snapshots.
2022-08-19 19:07:07 +02:00
Michael Eischer af50fe9ac0 mount: Map slashes in tags to underscores
Suggested-by: greatroar <>
2022-08-19 18:17:57 +02:00
Michael Eischer 0b7291b8b2 mount: Fix parent inode used by snapshots dir 2022-08-07 13:03:32 +02:00
greatroar cfa80e2c6b mount: remove unused inode field from root node 2022-08-07 13:03:26 +02:00
Michael Eischer caa17988a3 fuse: Redesign snapshot dirstruct
Cleanly separate the directory presentation and the snapshot directory
structure. SnapshotsDir now translates the dirStruct into a format
usable by the fuse library and contains only minimal special case rules.
All decisions have moved into SnapshotsDirStructure which now creates a
fully preassembled tree data structure.
2022-08-07 12:13:06 +02:00
Michael Eischer 2db7733ee3 fuse: remove unused MetaDir 2022-08-05 23:46:46 +02:00
Michael Eischer f678f7cb04 fuse: cleanup test 2022-08-05 23:46:46 +02:00
Alexander Weiss 1751afae26 Make snapshots dirs in mount command customizable 2022-08-05 23:46:46 +02:00
Alexander Weiss 57f4003f2f Generalize fuse snapshot dirs implemetation
+ allow "/" in tags and snapshot template
2022-08-05 23:46:46 +02:00
Michael Eischer fbcbd5318c repository: extract LoadTree/SaveTree
The repository has no real idea what a Tree is. So these methods never
belonged there.
2022-07-17 13:11:28 +02:00
Michael Eischer 3d29083e60 copy/find/ls/recover/stats: Memorize snapshot listing before index
These commands filter the snapshots according to some criteria which
essentially requires loading the index before filtering the snapshots.
Thus create a copy of the snapshots list beforehand and use it later on.
2022-04-09 12:26:30 +02:00
Michael Eischer c60540b196 add go:build headers everywhere 2022-03-28 22:23:47 +02:00
Jason Lenz 2b1932a258 Report symlink sizes from FUSE mount for snapshot dir
Fixes #3667.
2022-03-17 22:21:47 -05:00
greatroar fdc738fb70 Report symlink sizes from FUSE mount
Fixes #3667.
2022-03-13 16:48:35 +01:00
Uli Martens 718966a81a Move Blobcache into dedicated internal package 2021-09-24 15:45:00 +02:00
Alexander Neumann 04ca69cc78 Address issues reported by golint 2021-01-30 20:45:57 +01:00
Alexander Neumann 75f53955ee errcheck: Add error checks
Most added checks are straight forward.
2021-01-30 20:02:37 +01:00
Michael Eischer d05c88a5d6 fuse: Properly set uid/gid for directories
In #2584 this was changed to use the uid/gid of the root node. This
would be okay for the top-level directory of a snapshot, however, this
change also applied to normal directories within a snapshot. This
change reverts the problematic part and adds a test that directory
attributes are represented correctly.
2020-11-30 23:41:49 +01:00
Michael Eischer 9ffb698c8d Modernize context import in internal/fuse
Migrated using `go fix`.
2020-10-06 14:55:13 +02:00
Michael Eischer 37113282ca Merge a few variable declaration and initializations 2020-09-05 10:05:34 +02:00
Alexander Weiss f8316948d1 Optimize FUSE - make command `restic mount` faster and consume less memory
- Add Open() functionality to dir
- only access index for blobs when file is read
- Implement NodeOpener and put one-time file stuff there
- Add comment about locking as suggested by bazil.org/fuse

=> Thanks at Michael Eischer for suggesting the last two improvements
2020-07-28 23:01:18 +02:00
greatroar 07da61baee Remove blob size cache from restic mount 2020-07-25 19:39:14 +02:00
Lars Lehtonen 9ac90cf5cd
internal/fuse: fix dropped test error 2020-07-12 21:42:31 -07:00
greatroar 58719e1f47 Replace mount's per-file cache by a global LRU cache 2020-07-12 18:27:16 +02:00
greatroar d42c169458 Fix quadratic file reading in restic mount 2020-07-12 18:27:16 +02:00
greatroar 649cbec6c5 Simplify build tags for restic mount
This command can only be built on Darwin, FreeBSD and Linux
(and if we upgrade bazil.org/fuse, only FreeBSD and Linux:
https://github.com/bazil/fuse/issues/224).

Listing the few supported operating systems explicitly here makes
porting restic to new platforms easier.
2020-05-12 11:30:41 +02:00
greatroar be5a0ff59f Centralize buffer allocation and size checking in Repository.LoadBlob
Benchmark results for internal/repository:

name              old time/op    new time/op    delta
LoadTree-8           479µs ± 2%     478µs ± 1%   ~     (p=0.780 n=10+9)
LoadBlob-8          11.6ms ± 2%    11.6ms ± 1%   ~     (p=0.631 n=10+10)
LoadAndDecrypt-8    13.2ms ± 2%    13.3ms ± 3%   ~     (p=0.631 n=10+10)

name              old alloc/op   new alloc/op   delta
LoadTree-8          41.2kB ± 0%    41.2kB ± 0%   ~     (all equal)
LoadBlob-8          2.28kB ± 0%    2.28kB ± 0%   ~     (all equal)
LoadAndDecrypt-8    2.10MB ± 0%    2.10MB ± 0%   ~     (all equal)

name              old allocs/op  new allocs/op  delta
LoadTree-8             652 ± 0%       652 ± 0%   ~     (all equal)
LoadBlob-8            24.0 ± 0%      24.0 ± 0%   ~     (all equal)
LoadAndDecrypt-8      30.0 ± 0%      30.0 ± 0%   ~     (all equal)

name              old speed      new speed      delta
LoadBlob-8        86.2MB/s ± 2%  86.4MB/s ± 1%   ~     (p=0.594 n=10+10)
LoadAndDecrypt-8  75.7MB/s ± 2%  75.4MB/s ± 3%   ~     (p=0.617 n=10+10)
2020-04-23 10:04:20 +02:00
MichaelEischer 4f00564574
Merge pull request #2621 from greatroar/fixes
Some small fixes
2020-04-18 18:07:08 +02:00
MichaelEischer f77477129f
Merge pull request #2584 from greatroar/mount-cache-uid-gid
Cache uid and gid for top directories in internal/fuse
2020-04-18 17:45:14 +02:00
greatroar 072cf7b02d Fix debug messages in internal/fuse 2020-04-18 17:39:06 +02:00
Alexander Weiss 9a9101d144 Support specifying multiple host flags for various commands
The `dump`, `find`, `forget`, `ls`, `mount`, `restore`, `snapshots`,
`stats` and `tag` commands will now take into account multiple
`--host` and `-H` flags.
2020-02-26 22:17:59 +01:00
greatroar 1b502fa9ef Cache uid and gid for top directories in internal/fuse 2020-02-24 10:46:09 +01:00
Iku Iwasa f080142137 Support NetBSD without fuse 2018-06-16 15:55:04 +09:00
Jasper Lievisse Adriaanse 96311d1a2b Add support for illumos/Solaris
This does come without xattr/fuse support at this point.

NB: not hooking up the integration tests as restic won't compile without
    cgo with Go < 1.10.
2018-03-04 20:11:29 +00:00
Alexander Neumann 663c57ab4d debug: Remove manual Str() call Log() 2018-01-25 20:49:41 +01:00
Alexander Neumann 9c55e8d69c Merge pull request #1549 from MJDSys/more_index_lookup_avoids
More optimizations to avoid calling Index.Lookup()
2018-01-24 20:53:30 +01:00
Matthew Dawson df2c03a6a4
repository/master_index: Optimize Index.Lookup()
When looking up a blob in the master index, with several
indexes present in the master index, a significant amount of time
is spent generating errors for each failed lookup.  However, these
errors are often used to check if a blob is present, but the contents
are not inspected making the overhead of the error not useful.

Instead, change Index.Lookup (and Index.LookupSize) to instead return
a boolean denoting if the blob was found instead of an error.  Also change
all the calls to these functions to handle the new function signature.

benchmark                                            old ns/op     new ns/op     delta
BenchmarkMasterIndexLookupSingleIndex-6              820           897           +9.39%
BenchmarkMasterIndexLookupMultipleIndex-6            12821         2001          -84.39%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       5378          492           -90.85%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     17026         1649          -90.31%

benchmark                                            old allocs     new allocs     delta
BenchmarkMasterIndexLookupSingleIndex-6              9              9              +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            59             19             -67.80%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       22             6              -72.73%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     72             16             -77.78%

benchmark                                            old bytes     new bytes     delta
BenchmarkMasterIndexLookupSingleIndex-6              160           160           +0.00%
BenchmarkMasterIndexLookupMultipleIndex-6            3200          240           -92.50%
BenchmarkMasterIndexLookupSingleIndexUnknown-6       1232          48            -96.10%
BenchmarkMasterIndexLookupMultipleIndexUnknown-6     4272          128           -97.00%
2018-01-23 22:25:56 -05:00
Alexander Neumann b0c6e53241 Fix calls to repo/backend.List() everywhere 2018-01-21 21:15:09 +01:00
Alexander Neumann 97565d68a2 fuse/mount: Add option for snapshot template 2018-01-15 14:47:17 +01:00
Alexander Neumann f49f5c5903 fuse: Don't return an error for EOF 2018-01-13 10:14:10 +01:00
Alexander Neumann d89f2e5226 fuse: Remove debug log message 2018-01-13 10:10:07 +01:00
Alexander Neumann 3f7d85360a fuse: Only reload list of snapshots once per minute 2017-12-28 13:18:27 +01:00
Alexander Neumann 81a04656c5 fuse: Cleanup node name 2017-12-13 20:21:18 +01:00
Alexander Neumann 7092af6329 fuse: Handle sockets/fifos/devs correctly
Closes #1463
2017-12-03 17:25:00 +01:00
Tobias Klein b57ca64275 fuse: bugfix: ignore empty tags 2017-10-16 16:22:08 +02:00
Tobias Klein 7e4ce0dacc fuse mount: speedup 2017-10-09 10:26:56 +02:00