Commit Graph

15 Commits

Author SHA1 Message Date
Michael Eischer 3cd927d180 rclone: Give rclone time to finish before closing stdin pipe
Calling `Close()` on the rclone backend sometimes failed during test
execution with 'signal: Broken pipe'. The stdio connection closed both
the stdin and stdout file descriptors at the same moment, therefore
giving rclone no chance to properly send any final http2 data frames.

Now the stdin connection to rclone is closed first and will only be
forcefully closed after a timeout. In case rclone exits before the
timeout then the stdio connection will be closed normally.
2020-07-26 00:29:25 +02:00
Michael Eischer 8554332894 rclone: Close rclone side of stdio_conn pipes
restic did not notice when the rclone subprocess exited unexpectedly.

restic manually created pipes for stdin and stdout and used these for the
connection to the rclone subprocess. The process creating a pipe gets
file descriptors for the sender and receiver side of a pipe and passes
them on to the subprocess. The expected behavior would be that reads or
writes in the parent process fail / return once the child process dies
as a pipe would now just have a reader or writer but not both.

However, this never happened as restic kept the reader and writer
file descriptors of the pipes. `cmd.StdinPipe` and `cmd.StdoutPipe`
close the subprocess side of pipes once the child process was started
and close the parent process side of pipes once wait has finished. We
can't use these functions as we need access to the raw `os.File` so just
replicate that behavior.
2020-07-26 00:29:25 +02:00
greatroar 3e93b36ca4 Make rclone.New private 2020-07-26 00:28:45 +02:00
Alexander Neumann 3865b59716 rclone: Rework backend option parsing
This change allows passing no arguments to rclone, using `-o
rclone.args=""`. It is helpful when running rclone remotely via SSH
using a key with a forced command (via `command=` in `authorized_keys`).
2019-03-02 10:36:42 +01:00
Alexander Neumann 7ac683c360 rclone: Inject debug logger for HTTP 2018-10-21 19:58:40 +02:00
Alexander Neumann bfd923e81e rclone: Respect bandwith limits 2018-05-22 20:48:17 +02:00
Alexander Neumann 3f48e0e0f4 Add extra options to rclone
For details see https://github.com/restic/restic/pull/1657#issuecomment-377707486
2018-04-01 10:34:30 +02:00
Alexander Neumann c43c94776b rclone: Make concurrent connections configurable 2018-04-01 10:18:38 +02:00
Alexander Neumann 0b776e63e7 backend/rclone: Request random file name
When `/` is requested, rclone returns the list of all files in the
remote, which is not what we want (and it can take quite some time).
2018-04-01 10:18:38 +02:00
Alexander Neumann 737d93860a Extend first timeout to 60 seconds. 2018-04-01 10:18:38 +02:00
Alexander Neumann 4d5c7a8749 backend/rclone: Make sure rclone terminates 2018-04-01 10:18:38 +02:00
Alexander Neumann fc0295016a Address code review comments 2018-04-01 10:18:38 +02:00
Alexander Neumann 99b62c11b8 backend/rclone: Stop rclone in case of errors 2018-04-01 10:18:38 +02:00
Alexander Neumann 6d9a029e09 backend/rclone: Prefix all error messages 2018-04-01 10:18:38 +02:00
Alexander Neumann fe99340e40 Add rclone backend 2018-04-01 10:16:31 +02:00