This guide demonstrates the steps required to quickly gain access the eRSA data storage service and provides general information.
To access the eRSA data storage service, you will need:
- An eRSA account.
- A storage allocation.
Storage allocations are available from eRSA under various funding models. Contact our storage team for details.
1. Getting started
Step 1 : Connecting to the server
Storage is available via our access servers.
We currently support the following protocols for data storage access:
- SFTP - sftp.ersa.edu.au
- SCP - rsync.ersa.edu.au
- RSYNC - rsync.ersa.edu.au
For more details on how to access the storage, see our guides here:
FileZilla - a graphical SFTP client
Commandline Transfers - scp, sftp, pscp, psftp
OwnCloud - browser-based tool to sync, transfer, and share data
- RSYNC - syncing files
Step 2 : Logon
You will need to provide your eRSA username and password to access the data storage.
Step 3 : How do I find my data?
Storage allocations are stored in a top-level folder called “data”.
Inside the “data” folder, there are seperate directories for each allocation.
Allocation directories generally have what is called a “shortname”. An example of a shortname would be “physics” or “robinson”.
For example, for someone with access to the “smithlab” data storage allocation:
|-- data |-- physics |-- robinson |-- smithlab |-- put your data here
This directory is contained within the
/data directory, so the path to the directory is:
with ‘ smithlab ’ replaced by the name of your allocated directory.
If your eRSA username has been registered with eRSA by the data custodian for the allocation, you will have permission to read and write the contents of this directory (i.e. upload, download and modify data). Other users can see the folder name, but will not be able to access the contents.
If you aren’t sure what the shortname of your storage allocation is, contact our service desk for assistance.
2. General Information for Accessing Data Storage
There are several methods to transfer data between your local desktop copmputer and your remote storage. Some of these methods can also be used to transfer data between two different remote storage locations (e.g. from your data storage to your cloud computing storage).
eRSA’s remote storage uses Unix-based servers. This means that if your desktop computer is Mac OSX or Linux, data transfers can easily be coordinated using text commands. Windows users can get similar functionality by installing an application to create Unix compatible commands (such as Cygwin or PuTTY-pscp). The FileZilla desktop client is available for all major operating systems and is an easy method to transfer files for any user that doesn’t require automation of transfers via scripting.
Methods for Transferring Data
OwnCloud - a web browser client that makes it easy to transfer files, sync folders with your PC, and share data with anyone.
- A desktop graphical client for SFTP - There is a userguide for using the application “FileZilla”, but the applications WinSCP and CyberDuck can also be used. These clients create a ‘drag-and-drop’ interface for transferring files between your desktop and your remote storage. This should be the defalt starting method to use if you are not comfortable with the command-line.
Command-line file transfers - Use the
sftpcommand (or PuTTY-psftp for Windows) to access your remote data storage and upload or download data. The
scpcommand (or PuTTY-pscp for Windows users) can copy files between your local desktop and your remote data storage.
Sync a directory using the command-line application
rsync- Use rsync (userguide) to create a folder (directory) on your remote storage that can be synchronised to contain the same data as a specified folder on your computer. Any changes to the data in one of these folders can be mirrored in the other folder using the rsync command.
To access your data stored on eRSA’s servers, you must first authenticate yourself as the owner of the username provided to you by eRSA.
There are two authentication methods supported by eRSA.
- Password authentication - You enter your eRSA password when prompted. This is the simpler approach to implement and is the default for accessing eRSA storage allocations.
- Public key authentication - You use a “key” instead of a password, which requires some initial setup on the server end, but means you don’t need to enter a password on every access. This is very handy if you are automating data transfers.
More about public keys
A simple real-world example of public key authentication would be a bouncer at a club. If you were to walk up to a bouncer at the entrance of a VIP club and ask to enter, the bouncer would check if you are on the list. If you are on the list, he lets you in. If not, you are refused entry.
Public key - As the name implies, this key can be shared with other servers and is used to identify you based on your private key. Private key - This is a secret key that needs to be kept safe and shouldn’t be shared with anyone. Setting up key-based authentication is beyond the scope of this document. Please contact our Service Desk if you require assistance.
eRSA storage servers use a UNIX-based operating system. To access your eRSA data storage, you communicate with the servers using text commands (often called UNIX or shell commands) entered into a command-line shell such as “Terminal”.
If you choose to access the data storage using a desktop client such as FileZilla, you can drag and drop files in a graphical interface, and the program will convert your actions into text commands for you. This means that learning the text commands is not necessary.
Many users may prefer to use command-line tools to access the remote data storage. Text commands are, in fact, quicker and easier than using ‘point-and-click’ applications once you have become comfortable with their usage. There are also more features available using text commands that aren’t available using the desktop clients, such as setting up automated or batch file transfers using a script or transfers between two remote hosts.
Some UNIX tips:
- Folders are referred to as Directories
- There is a path to each file, that is written as
- The first
/in the path is the “root directory” which is the top-most directory in the hierarchy
.A single fullstop is a symbol meaning the current directory.
..Two full stops symbolises the directory which contains the current directory (one step up the hierarchical ladder).