Skip to content

Instantly share code, notes, and snippets.

@vingkan
Created August 23, 2024 19:18
Show Gist options
  • Save vingkan/70bfe89e50b5d932bde7116bc1ea2b8c to your computer and use it in GitHub Desktop.
Save vingkan/70bfe89e50b5d932bde7116bc1ea2b8c to your computer and use it in GitHub Desktop.
Split and combine a SQLite database file into smaller parts to stay under GitHub size limits.

Combine and Split a SQLite Database File

The first time you use this script, you will have to give it permissions:

chmod 0777 split_database.sh
chmod 0777 combine_database.sh

Then you can run the scripts like this:

./split_database.sh
./combine_database.sh
#!/bin/bash
# Concatenate the compressed database chunks into a single file
cat database_chunks/db-part-* > compressed.db.tgz
# Uncompress the combined file, which will output nfl_data.db
tar -xf compressed.db.tgz
# Delete the compressed file
rm compressed.db.tgz
#!/bin/bash
# Delete the existing database chunks folder, if it exists
rm -rf database_chunks
# Compress the database
tar cfz compressed.db.tgz nfl_data.db
# Create a new folder for the database chunks
mkdir database_chunks
# Split the compressed database into chunks of up to 10MiB each
split -b 10m compressed.db.tgz compressed/db-part-
# Delete the compressed file
rm compressed.db.tgz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment