KeiruaProd

Making encrypted cloud copies of your critical files

Problem

We could encrypt the files ourselves and rsync the files to another server we use, but there are options that do not require us to have an SSH access to another machine.

rclone

There are many tools to copy files from one server to another. I’ve used rclone a bit recently.

Also, it has a nice ande active forum where the project lead offers helpful answers.

Configuration

We’ll need to create 2 remotes:

In order to do this, we will need to run rclone config twice. rclone config updates the settings of rclone’s config file in ~/.config/rclone/rclone.conf.

It can look like this (~/.config/rclone/rclone.conf) : we will encrypt things stored in the backups bucket of the my-s3-provider remote.

# ~/.config/rclone/rclone.conf
[my-s3-provider]
type = s3
provider = …
access_key_id = …
secret_access_key = …
…

[secret]
type = crypt
remote = my-s3-provider:backups
filename_encryption = standard
directory_name_encryption = true
password = …
password2 = …

Protecting your credentials

Out of the box, rclone stores the credentials in plaintext in its config file, so anybody with read access to the file will have the credentials. What I find a bit better but still explicit is to store the credentials in an environment variable, and reference it rclone.conf:

secret_access_key = ${YOUR_S3_ACCESS_KEY}

If you have randow go stacktraces when you try to do anything, have a look at your credentials.

Synchronizing backups and download

Ok, so now we can send some files using rclone. We have various options:

Let’s create an encrypted-backups at the root of our bucket. Then, we can upload our backups in this directory:

rclone move --progress --s3-chunk-size=20M /your/backup/directory secret:/encrypted-backups --log-file=rclone-upload.log

When you’ll download a file, you’ll do it like this:

rclone copy --max-age 24h --progress secret:/encrypted-backups/ ./backups

When you want to list the content of your directory, you’ll need to use the secret proxy, unless you want the encrypted files:

rclone lsf my-s3-provider:backups/encrypted-backups
** a lot of random gibberish**
rclone lsf secret:/encrypted-backups
** the actual file names**

Going further

There are a lot of other commands, and they have many parameters. Usually, the --help flag can provide a lot of help. Then, knowing some shell script can be useful. Here is how to find the latest file:

rclone lsf --files-only --format "tp" secret:/encrypted-backups | sort | tail -1 | cut -d';' -f2

See a typo ? You can suggest a modification on Github.