File transfer to/out of the cluster¶
Remote sync (’rsync’) is a flexible network-enabled syncing tool. The syntax is very similar to ’cp’.
To sync the content of ’dir1’ to ’dir2’ on the same machine,
-r means recursive necessary for dir syncing:
rsync -r dir1/ dir2
Note the ’/’ after dir1. It means that you are synchronizing the content of the dir1, not the directory itself.
-a option is a combination flag. It's a quick way of saying you want recursion and want to preserve almost everything, except for hard links. The only exception to the preceding equivalence is when there is
--files-from, in which case
-r isn't implied.
--verbose increases the amount of information the daemon logs during its startup phase.
rsync perform a trial run that doesn't make any changes but produces mostly the same out‐put as a real run. It's most commonly used in combination with the
-i options to see what an
rsync command is going to do before one actually runs it.
Sync with a remote machine¶
- Push operation to push a dir to a remote machine:
rsync -av source_dir email@example.com:dest_dir
- Pull operation to pull a dir from a remote machine:
rsync -av firstname.lastname@example.org:source_dir ~/dest_dir
Useful options for rsync¶
-zoption reduces traffic by adding compression.
-Poption combines the flags
Secure copy (SCP) is a utility that allows you to securely copy files and directories between two locations.
You're able to copy files or directory:
- From/to your local machine to/out of a remote machine
- Between two remote systems
SCP encrypts both the files and passwords, so they're safe from any traffic snooping. The SCP depends on ssh for data transfer, therefore it requires an ssh key or password for the authentication. Below are basic examples.
Be extra careful copying files which share the same name and location on both machines, SCP overwrites files without any warnings.
Transferring large files, it's recommended to execute the SCP inside a ’screen’ session
The SCP syntax is:
scp [OPTION] user@SRC_HOST:path/to/file user@DEST_HOST:path/to/file
[OPTION]- scp option :
user@SRC_HOST:path/to/source/file- source file
user@DEST_HOST:path/to/destination/file- destination file
Copy a local file to a remote machine¶
scp path/to/local/file email@example.com:path/to/remote/dir
scp -r path/to/local/dir firstname.lastname@example.org:path/to/remote/dir
Copy a remote file to your local machine¶
scp email@example.com:path/to/local/file local/dir
Copy a file between two remote machines¶
Unlike ’rsync’, with ’scp’ you don't have to log into one of the servers to transfer file from one to another machine
scp firstname.lastname@example.org:path/to/file email@example.com:path/to/file
To route the traffic though your local machine, use
Secure or SSH File Transfer Protocol (SFTP) is method of transferring file between two remote system over a secure connection.
SFTP is inside of various graphical tools (see FileZilla for Windows below).
For a temporary access to certain files, you can request an SFTP account.
In order to start file transfer, you need to establish an SFTP session with a remote machine:
To see the syntax and how to use SFTP options, run
help in the session:
Navigating inside SFTP session¶
To navigate on your local machine,
l stands for local in front of a command. To get the current directory on the local machine:
Transferring remote files to your local machine¶
To download a remote file:
sftp> get testfile
sftp> get -Pr testdirectory
Transferring local files to your remote machine¶
To download a remote file:
sftp> put testfile
sftp> put -Pr testdirectory
How to connect using the FileZilla client¶
Having downloaded and installed the FileZilla client, enter the following and click ’Quickstart’ button.
- Username: ’<your username>’
- Password: ’<your password>’
If you have a passwordless connection configured with the server, you don't have to insert the password.
Insert the credentials and press ’Quickstart’ button
Accept the ’Unknown host key’
Files on your local machine are on the left hand side and remote files are on the right hand side. By clicking ’right’ mouse button on the file of interest, you can upload it to the other machine.