Skip to content

Instantly share code, notes, and snippets.

@byteshiva

byteshiva/gist:ebedab43b709c8bddfed16b4454a14b0

Created March 16, 2017 12:45
Show Gist options
  • Save byteshiva/ebedab43b709c8bddfed16b4454a14b0 to your computer and use it in GitHub Desktop.
Save byteshiva/ebedab43b709c8bddfed16b4454a14b0 to your computer and use it in GitHub Desktop.
Download ZIP
copy large files using dd command
Raw
gistfile1.txt
dd if=/dev/zero of=CREATE_TEST_FILE bs=1k count=4700000
@byteshiva
Copy link
Author

byteshiva commented Mar 16, 2017 •
edited
Loading

refe: http://linoxide.com/linux-command/linux-dd-command-create-1gb-file/

  1. https://linuxaria.com/howto/the-dd-command-on-linux-terminal
  2. https://www.ndchost.com/wiki/server-administration/netcat-over-ssh

@byteshiva
Copy link
Author

byteshiva commented Mar 16, 2017 •
edited
Loading

parallel faster data transmission form machine 1 to machine 2

bbcp -P 2 -w 2M -s 10 my.big.file dest@dest-ip-addr:/tmp/
  1. http://www.slac.stanford.edu/~abh/bbcp/
  2. http://intermediatesql.com/linux/scrap-the-scp-how-to-copy-data-fast-using-pigz-and-nc/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment