Skip to content

Instantly share code, notes, and snippets.

@peppyheppy
Forked from adeubank/chunked_mysql_dump.sh
Created August 14, 2014 23:50
Show Gist options
  • Save peppyheppy/29004aface9ff0083687 to your computer and use it in GitHub Desktop.
Save peppyheppy/29004aface9ff0083687 to your computer and use it in GitHub Desktop.
#!/bin/bash
# Connects to a MySQL database to dump data in batches.
# Great for dumping large datasets.
LIMIT=1000000
OFFSET=0
MAX_ROWS=1000000000
# This loops until it has read MAX_ROWS
while [ $OFFSET -lt $MAX_ROWS ]
do
echo "Fetching rows starting $OFFSET"
echo "SELECT COL_NAME FROM TABLE_NAME LIMIT $LIMIT OFFSET $OFFSET" | mysql DATABASE_NAME --skip-column-names -B -u MYSQLUSERNAME -pMYSQL_PASSWD >> output.csv
OFFSET=$((OFFSET+LIMIT))
done
# All you have to do is tweak the LIMIT and MAX variable to your liking.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment