Use of rsync (Remote Sync) command in Unix & Linux
rsync is a very useful alternative to rcp written by Andrew Tridgell and Paul Mackerras. This tool lets you copy files and directories between a local host and a remote host (source and destination can also be local if you need.) The main advantage of using rsync instead of rcp, is that rsync can use SSH as a secure channel, send/receive only the bytes inside files that changed since the last replication, and remove files on the destination host if those files were deleted on the source host to keep both hosts in sync. In addition to using rcp/ssh for transport, you can also use rsync itself, in which case you will connect to TCP port 873.
Whether you rely on SSH or use rsync explicitly, rsync still needs to be installed on both hosts. A Win32 port is available if you need, so you can have either one of the host or both be NT hosts. rsync's official web site has some good information and links.
Steps to configure rsync (Remote Sync) configuration file /etc/rsyncd.conf
Being co-written by Andrew Tridgell, author of Samba, it's no surprise that rsync's configuration file looks just like Samba and that rsync lets you create projects that look like shared directories under Samba. Accessing remote resources through this indirect channel offers more independence, as it lets you move files on the source rsync server without changing anything on the destination host.
Any parameters listed before any [module] section are global, default parameters. Each module is a symbolic name for a directory on the local host.
Here's an rsync configuration example:
#/etc/rsyncd.conf
secrets file = /etc/rsyncd.secrets
motd file = /etc/rsyncd.motd #Below are actually defaults, but to be on the safe side...
read only = yes
list = yes
uid = nobody
gid = nobody
[out]
comment = Great stuff from remote.acme.com
path = /home/rsync/out
[confidential]
comment = For your eyes only
path = /home/rsync/secret-out
auth users = joe,jane
hosts allow = *.acme.com
hosts deny = *
list = false
Note: rsync will not grant access to a protected share if the password file (/etc/rsyncd.secrets, here) is world-readable.Running
The below steps will explain how to start rsync service and use it in details.
The rsync daemon is launched by specifying the --daemon option to rsync. You can launch it either via inetd or as a stand-alone daemon. When run via inetd you should add a line like this to /etc/services:
rsync 873/tcp
and a single line something like below to /etc/inetd.conf:
rsync stream tcp nowait root /usr/bin/rsync rsyncd --daemon
You will then need to send inetd a HUP signal to tell it to reread its config file. Note that you should not send the rsync server a HUP signal to force it to reread the /etc/rsyncd.conf. The file is re-read on each client connection.
Per the HOWTO:
The rsync daemon is robust, so it is safe to launch it as a stand-alone server. The code that loops waiting for requests is only a few lines long then it forks a new copy. If the forked process dies then it doesn't harm the main daemon. The big advantage of running as a daemon will come when the planned directory cache system is implemented. The caching system will probably only be enable when running as a daemon. For this reason, busy sites is recommended to run rsync as a daemon. Also, the daemon mode makes it easy to limit the number of concurrent connections.
Since it's not included in the 2.4.3 RPM package, here's the init script to be copied as /etc/rc.d/init.d/rsyncd with symlinks to /etc/rc.d/rc3.d:
#!/bin/sh
# Rsyncd This shell script takes care of starting and stopping the rsync daemon
# description: Rsync is an awesome replication tool.
# Source function library. . /etc/rc.d/init.d/functions
[ -f /usr/bin/rsync ] || exit 0
case "$1" in
start) action "Starting rsyncd: " /usr/bin/rsync --daemon
;;
stop)
action "Stopping rsyncd: " killall rsync
;;
*)
echo "Usage: rsyncd {start|stop}"
exit 1
esac
exit 0
Here's an example under Linux on how to set up a replication through SSH:
rsync -avz -e ssh rsync@remote.acme.com:/home/rsync/out/ /home/rsync/from_remote
An important thing here, is that the presence or absence of a trailing "/" in the source directory determines whether the directory itself is copied, or simply the contents of this source directory.
In other words, the above means that the local host must have a directory available (here, /home/rsync/from_remote to receive the contents of /home/rsync/out sitting on the remote host, otherwise rsync will happily download all files into the path given as destination without asking for confirmation, and you could end up with a big mess.
On the other hand, rsync -avz -e ssh rsync@remote.acme.com:/home/rsync/out /home/rsync/from_remote means that the an "out" sub-directory is first created under /home/rsync/from_remote on the destination host, and will be populated with the contents of the remote directory ./out. In this case, files will be save on the local host in /home/rsync/from_remote/out, so the former commands looks like a better choice.
Here's how to replicate an Rsync share from a remote host:
rsync -avz rsync@remote.acme.com::out /home/rsync/in
Notice that we do not use a path to give the source resource, but instead just a name ("out"), and that we use :: to separate the server's name and the resource it offers. In the rsync configuration that we'll see just below, this is shown as a [out] section. This way, admins on remote.acme.com can move files on their server; As long as they remember to update the actual path in the [out] section (eg. PATH=/home/rsync/out to PATH=/home/outgoing), remote rsync users are not affected.
An rsync server displays the list of available anonymous shares through rsync remote.acme.com::. Note the ::. For added security, it is possible to prompt for a password when listing private shares, so that only authorized remote users know about the rsync shares available from your server.
If any feedback, queries are always welcome!
Thank you for posting this. I quite agree with your opinion.
nice explanation but should we use scp or rsync while copying from one server to another ? which is the good ?