Skip to content

Instantly share code, notes, and snippets.

@wardbekker
Created May 10, 2011 09:16
Show Gist options
  • Save wardbekker/964146 to your computer and use it in GitHub Desktop.
Save wardbekker/964146 to your computer and use it in GitHub Desktop.
Naive parallel import of Compressed MYSQL dump file
# Split MYSQL dump file
zcat dump.sql.gz | awk '/DROP TABLE IF EXISTS/{n++}{print >"out" n ".sql" }'
# Parallel import using GNU Parallel http://www.gnu.org/software/parallel/
ls -rS *.sql | parallel --joblog joblog.txt mysql -uXXX -pYYY db_name "<"
@SchizoDuckie
Copy link

Works brilliant with 10 threads parallel:

ls -rS data.*.sql | parallel -j10 --joblog joblog.txt mysql -uuser -ppass dbname "<"

@wardbekker
Copy link
Author

OSX compatible:

gunzip -c wiebetaaltwat_stable.sql.gz | awk '/DROP TABLE IF EXISTS/{n++}{filename = "out" n ".sql"; print > filename}'

@vojkny
Copy link

vojkny commented Oct 17, 2018

mysql imports are sequential and this will not have any effect.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment