Skip to content

Instantly share code, notes, and snippets.

@illucent
Forked from L422Y/multisync.sh
Created November 22, 2015 11:21
Show Gist options
  • Select an option

  • Save illucent/7011fbef8d19273653b6 to your computer and use it in GitHub Desktop.

Select an option

Save illucent/7011fbef8d19273653b6 to your computer and use it in GitHub Desktop.
lftp script to mirror remote ftp site to local folder using multiple connections per file and parallel file downloads (for reference/cron jobs)
#!/bin/bash
do_multisync() {
remote_dir="private/files"
local_dir="/root/i_drive/files"
# remote host
host="ftp.myremotehost.com"
# username / password
user="larry"
pass="thisisnotmypassword"
# number of connections / parts per file transfer
partsperfile=10
# number of parallel transfers at a time
numfiles=2
# get number of active multisync processes
numprocs=`ps aux | grep "[m]ultisync\$" | wc -l`
if [[ $numprocs -gt 2 ]] ; then
echo "Previous 'multisync' is still running. ($numprocs)"
exit 1
else
echo "Running..."
(
echo open ${host}
echo user ${user} ${pass}
echo mirror -v -c --use-pget-n=${partsperfile} --parallel=${numfiles} ${remote_dir} ${local_dir}
echo bye
) | lftp -f /dev/stdin
exit 0
fi
}
do_multisync >> /tmp/multisync.log &
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment