Contents


Introduction

Welcome to the rsync user guide. This guide details the steps used to access eRSA storage services via use of the rsync command. Rsync is a file synchronization and file transfer command for Unix-like systems that minimizes network data transfer. It can also compress and encrypt data streams. The rsync command is available on most Linux distributions, Mac OSX and can be installed on a Windows machine.

Use cases for rsync

Generally, rsync is used to synchronise the contents of two folders- a source folder and a destination folder. The source and destination folders can be on the same computer, or they could be on separate computers/servers that are accessible over a network or the internet. Rsync uses smart algorithms to transfer data efficiently. If you were to rsync 50GB of data from your desktop to a remote server, then the first time the rsync runs, the entire 50GB is copied. If at a later stage you add an additional 1GB of data and run the rsync again, it will copy only the new and changed files. So, in the above case, only the new 1GB of data would be transferred, preserving the existing 50GB. Very handy!

eRSA support of rsync

eRSA currently supports “rsync over ssh” only. This means

  • For security reasons, we support only encrypted transfers (using SSH or secure shell). No plain, unencrypted pipes.
  • We require that the user authenticates (explained later) before a data transfer can occur.
Installation of rsync
  • Under Linux, rsync generally is installed using a package manager. For example, apt-get or yum.
  • On Mac OSX, rsync is available by default.
  • Under Windows, rsync can be installed as part of the cygwin package and used from the command line. Please note that installation of cygwin is outside the scope of eRSA support.

Steps to use rsync

Authentication Methods

There are two authentication methods supported by eRSA.

  • Password authentication - You enter a password when prompted. This is the simpler approach to implement.
  • Public key authentication - You use a “key” instead of a password, which requires some setup on the server end, but means you don’t need to enter a password on every access. Very handy if you are automating data transfers.
More about public keys

A simple real-world example of public key authentication would be a bouncer at a club. If you were to walk up to a bouncer at the entrance of a VIP club and ask to enter, the bouncer would check if you are on the list. If you are on the list, he lets you in. If not, you are refused entry.

Rsync over SSH uses public and private keys that are generated on your desktop.

  • Public key - As the name implies, this key can be shared with other servers and is used to identify you based on your private key.
  • Private key - This is a secret key that needs to be kept safe and shouldn’t be shared with anyone.

Setting up key-based authentication is beyond the scope of this document. Please contact our Service Desk if you require assistance.

Running the rsync command

Regardless of the authentication method used, the process of copying files with rsync is identical.

  • Rsync requires a couple of flags to operate. The most common are  -a  and  -v .
    •  -a  meaning “archive mode”.
    •  -v  meaning “verbose mode”. This gives you more information about what rsync is doing.
    •  -z  meaning “compress data”. Rsync will attempt to compress the data it transfers, which can save data bandwidth. The amount of compression attained depends on the type of data being transferred.
    • Another handy flag is the  -n  flag. This is the dry-run setting, which allows you to test a file copy, without copying any files.
    • For details on all the available flags, Run the command  rsync --help .
  • Next, you need to specify a source and destination to copy to and from.
  • If the source or destination is on a remote system, you will need to tell rsync what username to use, and the hostname or IP address of the remote system.

Transferring files to and from eRSA

Copying files to eRSA

  • If you had a directory called “stuff” on your local system and wanted to copy or rsync it to your storage allocation (for example “myresearchgroup”), here is how to do so:
    1. To perform this step, you will require an eRSA username and password along with a valid storage allocation. If public keys have been setup, no password is needed.
    2.  rsync -av stuff/ username@rsync.ersa.edu.au:/data/myresearchgroup/stuff 
    3. You will either be prompted for a password, or your public key will be checked.
    4. This will copy the contents of the local folder called “stuff” to the remote eRSA server, “rsync.ersa.edu.au” and put these contents on the remote server under the folder “/data/myresearchgroup/stuff”

Copying files from eRSA

  • To copy files from the remote server onto your local machine:
  1. To perform this step, you will require an eRSA username and password along with a valid storage allocation. If public keys have been setup, no password is needed.
  2.  rsync -av username@rsync.ersa.edu.au:/data/myresearchgroup/stuff/stuff 
  3. You will either be prompted for a password, or your public key will be checked.
  4. This will copy the contents from the remote eRSA server, “rsync.ersa.edu.au” from the remote folder “data/myresearchgroup/stuff” to a local folder called “stuff”.


Did you find this information useful? Share your feedback here.