Lsync tutorial

lsync tutorial

 Using lsyncd to synchronize file changes across multiple hosts
Posted 07-18-22 at 06:00 PM by ychaouche
Updated 05-30-23 at 07:12 AM by ychaouche
Tags live synchronization, lsyncd, rsync
In my own machine,
I create a directory named SYNC.
I put all the files I need to synchronize with other machines in there,
and have their original locations be symlinks to their new location inside the SYNC directory.



Code:

    $ mkdir SYNC
    $ mv .bashrc_common SYNC/
    $ ln -s ~/SYNC/.bashrc 
    $ mv .emacs.d/init.el SYNC/
    $ ln -s ~/SYNC/init.el .emacs.d/

These files are useful to me for any of the servers I manage.
I'd like to get changes immediately reflected on other servers
via automatic synchronization
done in the background,
so that I don't have to worry about copying my files around each time they're modified.

By configuring lsyncd to monitor this directory for changes,
I just set it up once and forget it.
When I add new files,
or modify old ones,
they get automatically picked up and copied to remote servers.

The synchronization is done via a simple configuration file.
The convention is to name the file lsyncd.conf
and it usually lives in /etc/.
At its minimum,
it consists of only two parts:
a global settings block
and multiple sync blocks,
one for each remote host.



Code:

     settings {
        logfile        = "/var/log/lsyncd.log",
        statusFile     = "/var/log/lsyncd.status",
        insist	  = true,
        nodeamon	  = true
     }

     sync {
       default.rsyncssh,     
       source	= "/home/ychaouche/SYNCHRO",
       host		= "10.10.10.19",
       targetdir	= "/root/SYNCHRO",
       ssh = {
          port  = "1234"
        }
     }    

In the settings,
I set its log and status files,
then instruct it to keep trying (insist) to sync a remote host if it's not responding
(it may be a temporary network issue).

Following are each remote host's sync block.

default.rsyncssh is the synchronization method we want to use.
We simply instruct lsyncd what directory to monitor (source),
the remote host to connect to,
the target directory where to put files,
then supply any ssh options in the ssh sub-block.

That's it.
Run lsyncd as root and let it do its work.

After approximately 7 seconds,
the SYNC folder is created on each remote server.
I only need to replace the original files with symlinks to their updated version inside of SYNC :


Code:

    $ mv .bashrc_common .bashrc_common-
    $ ln -s /root/SYNC/.bashrc_common
    $ mv .emacs.d/init.el .emacs.d/init.el-
    $ ln -s /root/SYNC/init.el .emacs.d/

I only do this once on each server.

Now,
anytime any of my files in SYNC/ get changed,
all of my servers are transparently updated in under 10 seconds.

Lsyncd is very nice.
It is very easy to setup,
a kind of "setup and forget" tool,
and it syncs very rapidly
without using any cronjob
via the inotify mechanism
(on linux, other machanisms are used on other systems).

Some things to consider when choosing lsyncd :

- naming conventions aren't consistent:
targetdir,
all lower letters,
but statusFile,
with capital F.

- doesn't watch single files:
it only knows about monitoring whole directories.

- poor logging information:
the default logs don't mention what host is implied for the log lines.

- can't have common config among hosts:
for example,
if you want to apply the same rsync options to all your hosts,
you'll have to manually copy/paste it across all your sync blocks.
There's no such thing as "sync-global" for eg.


To learn more about it,
see official documentation here : https://axkibe.github.io/lsyncd/