Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for rclone:// repositories (via borgstore) #8405

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -411,6 +411,7 @@ Due to using the `borgstore` project, borg now also supports other kinds of

- sftp: the borg client will directly talk to an sftp server.
This does not require borg being installed on the sftp server.
- rclone: the borg client will talk via rclone to cloud storage.
- Others may come in the future, adding backends to `borgstore` is rather simple.

Restoring a backup
Expand Down
2 changes: 1 addition & 1 deletion docs/usage/general/file-systems.rst.inc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Pros:
and re-write segment files to free space.
- In future, easier to adapt to other kinds of storage:
borgstore's backends are quite simple to implement.
A ``sftp:`` backend already exists, cloud storage might be easy to add.
``sftp:`` and ``rclone:`` backends already exists, others might be easy to add.
- Parallel repository access with less locking is easier to implement.

Cons:
Expand Down
9 changes: 7 additions & 2 deletions docs/usage/general/repository-urls.rst.inc
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,23 @@ Note: you may also prepend a ``file://`` to a filesystem path to get URL style.

**Remote repositories** accessed via ssh user@host:

``ssh://user@host:port/path/to/repo`` - absolute path`
``ssh://user@host:port/path/to/repo`` - absolute path

``ssh://user@host:port/./path/to/repo`` - path relative to current directory

``ssh://user@host:port/~/path/to/repo`` - path relative to user's home directory

**Remote repositories** accessed via sftp:

``sftp://user@host:port/path/to/repo`` - absolute path`
``sftp://user@host:port/path/to/repo`` - absolute path

For ssh and sftp URLs, the ``user@`` and ``:port`` parts are optional.

**Remote repositories** accessed via rclone:

``rclone://remote:path``


If you frequently need the same repo URL, it is a good idea to set the
``BORG_REPO`` environment variable to set a default for the repo URL:

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ classifiers = [
]
license = {text="BSD"}
dependencies = [
"borgstore ~= 0.0.1",
"borgstore ~= 0.0.4",
"msgpack >=1.0.3, <=1.1.0",
"packaging",
"platformdirs >=3.0.0, <5.0.0; sys_platform == 'darwin'", # for macOS: breaking changes in 3.0.0,
Expand Down
2 changes: 1 addition & 1 deletion src/borg/archiver/_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ def get_repository(
args=args,
)

elif location.proto in ("sftp", "file") and not v1_or_v2: # stuff directly supported by borgstore
elif location.proto in ("sftp", "file", "rclone") and not v1_or_v2: # stuff directly supported by borgstore
repository = Repository(
location,
create=create,
Expand Down
15 changes: 14 additions & 1 deletion src/borg/helpers/parseformat.py
Original file line number Diff line number Diff line change
Expand Up @@ -467,6 +467,14 @@ class Location:
re.VERBOSE,
) # path

rclone_re = re.compile(
r"""
(?P<proto>rclone):// # rclone://
(?P<path>(.*))
""",
re.VERBOSE,
) # path

socket_re = re.compile(
r"""
(?P<proto>socket):// # socket://
Expand Down Expand Up @@ -546,6 +554,11 @@ def normpath_special(p):
self.port = m.group("port") and int(m.group("port")) or None
self.path = normpath_special(m.group("path"))
return True
m = self.rclone_re.match(text)
if m:
self.proto = m.group("proto")
self.path = m.group("path")
return True
m = self.file_re.match(text)
if m:
self.proto = m.group("proto")
Expand Down Expand Up @@ -575,7 +588,7 @@ def __str__(self):

def to_key_filename(self):
name = re.sub(r"[^\w]", "_", self.path).strip("_")
if self.proto not in ("file", "socket"):
if self.proto not in ("file", "socket", "rclone"):
name = re.sub(r"[^\w]", "_", self.host) + "__" + name
if len(name) > 100:
# Limit file names to some reasonable length. Most file systems
Expand Down
8 changes: 8 additions & 0 deletions src/borg/testsuite/helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,14 @@ def test_ssh(self, monkeypatch, keys_dir):
"host='2a02:0001:0002:0003:0004:0005:0006:0007', port=1234, path='/some/path')"
)

def test_rclone(self, monkeypatch, keys_dir):
monkeypatch.delenv("BORG_REPO", raising=False)
assert (
repr(Location("rclone://remote:path"))
== "Location(proto='rclone', user=None, host=None, port=None, path='remote:path')"
)
assert Location("rclone://remote:path").to_key_filename() == keys_dir + "remote_path"

def test_sftp(self, monkeypatch, keys_dir):
monkeypatch.delenv("BORG_REPO", raising=False)
assert (
Expand Down
Loading