Skip to content

Instantly share code, notes, and snippets.

@jihao
Created February 25, 2015 06:47
Show Gist options
  • Save jihao/242b6695079d51ac2581 to your computer and use it in GitHub Desktop.
Save jihao/242b6695079d51ac2581 to your computer and use it in GitHub Desktop.
sync file to hadoop slaves nodes using scp
#!/bin/bash
slaves=(`cat $HADOOP_PREFIX/etc/hadoop/slaves`)
for slave in ${slaves[@]}
do
echo "kill HRegionServer in $slave ..."
`ssh $USER@$slave "jps | grep HRegionServer | awk '{print \$1}' | xargs kill -9"`
done
#!/bin/bash
slaves=(`cat $HADOOP_PREFIX/etc/hadoop/slaves`)
for file in "$@"
do
fullpath=`readlink -f "$file"`
for slave in ${slaves[@]}
do
echo "sync $fullpath with $slave ..."
`scp $fullpath $USER@$slave:$fullpath`
done
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment