rdelaage
/
restic
mirror of https://github.com/restic/restic.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,838 Commits 7 Branches 44 Tags 74 MiB
Commit Graph

1175 Commits

Author SHA1 Message Date
Michael Eischer cd783358d3 local: Limit concurrent backend operations 
Use a limit of 2 similar to the filereader concurrency in the archiver.
 2022-04-09 12:21:38 +02:00
Michael Eischer 0b258cc054 backends: clean reader closing 2022-04-09 12:21:38 +02:00
Charlie Jiang d9c9415cfd Fix rclone (scoop shim) and sftp issue due to detached console on Windows 2022-04-03 17:53:17 +08:00
greatroar c23c0f7c14 Cast unix.Statfs_t.Type to int64 when checking for btrfs 
Fixes #3687. Uses the cast suggested by @MichaelEischer, except that the
contant isn't cast along, because it's untyped and will be converted by
the compiler as necessary.
 2022-03-31 22:30:45 +02:00
Alexander Neumann db8a958991
Merge pull request #3683 from MichaelEischer/fix-golangci-lint-warnings 
Fix golangci lint warnings
 2022-03-29 11:45:10 +02:00
Michael Eischer af31266b7d golangci-lint: replace deprecated golint with revive 2022-03-28 22:33:17 +02:00
Michael Eischer 2f81af6afa bloblru: Fix comment for New function 2022-03-28 22:25:25 +02:00
Michael Eischer 61e179ee78 switch to golang.org/x/term 2022-03-28 22:24:15 +02:00
Michael Eischer c60540b196 add go:build headers everywhere 2022-03-28 22:23:47 +02:00
Michael Eischer fefe9f5c0e pack: Hide more implementation details 2022-03-28 22:12:16 +02:00
Michael Eischer a773cb6527 pack: cleanup header size calculation 2022-03-28 22:09:49 +02:00
Michael Eischer 6408686973 repository: Simplify Blob equality check 2022-03-28 22:09:49 +02:00
Michael Eischer 243698680a crypto: Use helpers for size calculations 2022-03-28 22:09:49 +02:00
Michael Eischer d6db5a1fc2 archiver: Fix test 
The test relied on an undeocumented sideeffect of the LoadBlob implementation
 2022-03-28 22:09:49 +02:00
Michael Eischer f78bd14e28 repository: Remove pack implementation details from MasterIndex 2022-03-28 22:09:49 +02:00
Michael Eischer dc3d77dacc repository: make saveAndEncrypt private 2022-03-28 22:09:49 +02:00
Michael Eischer 6877e7edbb repository: Rename LoadAndDecrypt to LoadUnpacked 
The method is the complement for SaveUnpacked and not for
SaveAndEncrypt. The latter assembles blobs into pack files.
 2022-03-28 22:09:49 +02:00
Michael Eischer 2e1613d4c6 errors: Ensure that errors.IsFatal(errors.Fatal("err")) == true 
This fixes a few cases where restic output "Fatal: Fatal: [...]"
 2022-03-28 22:09:49 +02:00
Michael Eischer 537b4c310a copy: Implement by reusing repack 
The repack operation copies all selected blobs from a set of pack files
into new pack files. For prune the source and destination repositories
are identical. To implement copy, just use a different source and
destination repository.
 2022-03-26 20:47:15 +01:00
Alexander Neumann 4d5db61bd0
Merge pull request #3484 from MichaelEischer/stream-check-repack 
Stream packs in `check --read-data` and during repacking
 2022-03-26 20:46:17 +01:00
Alexander Neumann e682f7c0d6 Add tests for StreamPack 2022-03-21 21:15:03 +01:00
Alexander Neumann 0b8b524f12
Merge pull request #3512 from MichaelEischer/cleaner-lock-refresh 
Prevent lock refresh from leaving behind lots of stale locks
 2022-03-21 11:10:37 +01:00
Alexander Neumann a350625554
Merge pull request #3524 from MichaelEischer/atomic-sftp 
sftp: Implement atomic uploads
 2022-03-21 11:08:22 +01:00
Alexander Neumann 8388f66c4c
Merge pull request #3668 from greatroar/symlink-size 
Report symlink sizes from FUSE mount
 2022-03-21 11:02:32 +01:00
Alexander Neumann 0937008648
Merge pull request #3654 from MichaelEischer/limit-huge-tree-streams 
Limit number of large tree blobs loaded in parallel by StreamTrees
 2022-03-21 11:01:04 +01:00
Michael Eischer cd190bee14 filter: short circuit if no negative patterns 2022-03-20 13:33:08 +01:00
Vincent Bernat 2ee07ded2b filter: ability to use negative patterns 
This is quite similar to gitignore. If a pattern is suffixed by an
exclamation mark and match a file that was previously matched by a
regular pattern, the match is cancelled. Notably, this can be used
with `--exclude-file` to cancel the exclusion of some files.

Like for gitignore, once a directory is excluded, it is not possible
to include files inside the directory. For example, a user wanting to
only keep `*.c` in some directory should not use:

    ~/work
    !~/work/*.c

But:

    ~/work/*
    !~/work/*.c

I didn't write documentation or changelog entry. I would like to get
feedback if this is the right approach for excluding/including files
at will for backups. I use something like this as an exclude file to
backup my home:

    $HOME/**/*
    !$HOME/Documents
    !$HOME/code
    !$HOME/.emacs.d
    !$HOME/games
    # [...]
    node_modules
    *~
    *.o
    *.lo
    *.pyc
    # [...]
    $HOME/code/linux/*
    !$HOME/code/linux/.git
    # [...]

There are some limitations for this change:

 - Patterns are not mixed accross methods: patterns from file are
   handled first and if a file is excluded with this method, it's not
   possible to reinclude it with `--exclude !something`.

 - Patterns starting with `!` are now interpreted as a negative
   pattern. I don't think anyone was relying on that.

 - The whole list of patterns is walked for each match. We may
   optimize later by exiting early if we know no pattern is starting
   with `!`.

Fix #233
 2022-03-20 13:33:08 +01:00
Michael Eischer 12606b575f filter: Cleanup variable naming 2022-03-20 13:33:08 +01:00
Michael Eischer 5f145f0c7e filter: introduce pattern struct 2022-03-20 13:33:08 +01:00
Vincent Bernat 13c40d4199 filter: additional tests for filter.List() 2022-03-20 13:33:08 +01:00
Jason Lenz 2b1932a258 Report symlink sizes from FUSE mount for snapshot dir 
Fixes #3667.
 2022-03-17 22:21:47 -05:00
greatroar fdc738fb70 Report symlink sizes from FUSE mount 
Fixes #3667.
 2022-03-13 16:48:35 +01:00
Michael Eischer 254c8743fc Limit number of large tree blobs loaded in parallel by StreamTrees 
Load tree blobs with more than 50MB only from a single goroutine. Very
large tree blobs with for example 400 MB size can otherwise require
roughly 1GB * streamTreeParallelism memory.
 2022-02-19 12:26:09 +01:00
Michael Eischer 27524979e8 restorer: Remove dead code 2022-02-13 11:43:09 +01:00
gum3ng dd30083c2b [#issue 3127] Add xattr support for Solaris 2022-02-13 14:24:37 +05:30
MichaelEischer fb4c5af5c4
Merge pull request #3642 from gco/master 
Fix test failures on Solaris
 2022-02-12 22:07:14 +01:00
Michael Eischer 5ec312ca06 sftp: Implement atomic uploads 
Create a temporary file with a sufficiently random name to essentially
avoid any chance of conflicts. Once the upload has finished remove the
temporary suffix. Interrupted upload thus will be ignored by restic.
 2022-02-12 20:30:49 +01:00
Michael Eischer c6fd13425b remember the refreshed lock file even if removal failed 
This ensures that restic won't create lots of new lock files without
deleting them later on.

In some cases a Delete operation on a backend can return a "File does
not exist" error even though the Delete operation succeeded. This can
for example be caused by request retries. This caused restic to forget
about the new lock file and continue trying to remove the old (already
deleted) lock file.
 2022-02-12 20:21:58 +01:00
Michael Eischer bba8ba7a5b repository: cancel streampack context after error 2022-02-12 20:18:25 +01:00
Michael Eischer 47554a3428 repository: Fix error handling in repack 
When storing a blob fails, this is a fatal error which must not be
retried.
 2022-02-12 20:18:25 +01:00
Michael Eischer 4b3dc415ef checker: cleanup header extraction 2022-02-12 20:18:25 +01:00
Michael Eischer 930a00ad54 checker: reuse bufio reader 2022-02-12 20:18:25 +01:00
Michael Eischer 34ebafb8b6 repository: don't crash if blob size is too short 2022-02-12 20:18:25 +01:00
Michael Eischer becebf5d88 repository: remove unused DownloadAndHash 2022-02-12 20:18:25 +01:00
Michael Eischer f1e58e7c7f checker: rewrite ReadData to stream packs 2022-02-12 20:18:25 +01:00
Michael Eischer f40abd92fa restorer: convert to use StreamPack 2022-02-12 20:18:25 +01:00
Michael Eischer f00f690658 repository: stream packs during repacking 2022-02-12 20:18:25 +01:00
Michael Eischer c4a2bfcb39 repository: Add StreamPacks function 
The function supports efficiently loading a specified list of blobs from
a single pack in a streaming fashion. That is there's no need for
temporary files independent of the pack size.
 2022-02-12 20:18:25 +01:00
Michael Eischer 153e2ba859 repository: Implement lisiting blobs per pack file 2022-02-12 20:18:24 +01:00
MichaelEischer cc90f2ba6b
Merge pull request #2816 from greatroar/noatime 
Set O_NOATIME flag on Linux
 2022-02-07 21:38:31 +01:00