readme.md

Back up and Restore an IndexedDB Database

This gist provides functions to import and export data from an IndexedDB database as JSON. It's based on Justin Emery's indexeddb-export-import package, but applies some adjustments that reflect better on the current browser landscape (i.e. better developer ergonomics but no support for Internet Explorer).

Usage

For each of the provided functionalities, you need a connected IDBDatabase instance.

Export Data

import { idb } from 'some-database'

import { exportToJson } from 'idb-backup-and-restore.js'

exportToJson(idb)
  .then(result => {
    console.log('Exported JSON string:', result)
  })
  .catch(error => {
    console.error('Something went wrong during export:', error)
  })

Import Data

import { idb } from 'some-database'
import { serializedData } from 'some-serialized-data'

import { importFromJson } from 'idb-backup-and-restore.js'

importFromJson(idb, serializedData)
  .then(() => {
    console.log('Successfully imported data')
  })
  .catch(error => {
    console.error('Something went wrong during import:', error)
  })

Clear Database

Depending on your use case, it can be reasonable to clear a database before importing serialized data:

import { idb } from 'some-database'
import { serializedData } from 'some-serialized-data'

import { importFromJson, clearDatabase } from 'idb-backup-and-restore.js'

clearDatabase(idb)
  .then(() => importFromJson(idb, serializedData))
  .then(() => {
    console.log('Successfully cleared database and imported data')
  })
  .catch(error => {
    console.error('Could not clear & import database:', error)
  })

idb-backup-and-restore.js

/**
 * Export all data from an IndexedDB database
 *
 * @param {IDBDatabase} idbDatabase The database to export from
 * @return {Promise<string>}
 */
export function exportToJson(idbDatabase) {
  return new Promise((resolve, reject) => {
    const exportObject = {}
    if (idbDatabase.objectStoreNames.length === 0) {
      resolve(JSON.stringify(exportObject))
    } else {
      const transaction = idbDatabase.transaction(
        idbDatabase.objectStoreNames,
        'readonly'
      )

      transaction.addEventListener('error', reject)

      for (const storeName of idbDatabase.objectStoreNames) {
        const allObjects = []
        transaction
          .objectStore(storeName)
          .openCursor()
          .addEventListener('success', event => {
            const cursor = event.target.result
            if (cursor) {
              // Cursor holds value, put it into store data
              allObjects.push(cursor.value)
              cursor.continue()
            } else {
              // No more values, store is done
              exportObject[storeName] = allObjects

              // Last store was handled
              if (
                idbDatabase.objectStoreNames.length ===
                Object.keys(exportObject).length
              ) {
                resolve(JSON.stringify(exportObject))
              }
            }
          })
      }
    }
  })
}

/**
 * Import data from JSON into an IndexedDB database.
 * This does not delete any existing data from the database, so keys may clash.
 *
 * @param {IDBDatabase} idbDatabase Database to import into
 * @param {string}      json        Data to import, one key per object store
 * @return {Promise<void>}
 */
export function importFromJson(idbDatabase, json) {
  return new Promise((resolve, reject) => {
    const transaction = idbDatabase.transaction(
      idbDatabase.objectStoreNames,
      'readwrite'
    )
    transaction.addEventListener('error', reject)

    var importObject = JSON.parse(json)
    for (const storeName of idbDatabase.objectStoreNames) {
      let count = 0
      for (const toAdd of importObject[storeName]) {
        const request = transaction.objectStore(storeName).add(toAdd)
        request.addEventListener('success', () => {
          count++
          if (count === importObject[storeName].length) {
            // Added all objects for this store
            delete importObject[storeName]
            if (Object.keys(importObject).length === 0) {
              // Added all object stores
              resolve()
            }
          }
        })
      }
    }
  })
}

/**
 * Clear a database
 *
 * @param {IDBDatabase} idbDatabase The database to delete all data from
 * @return {Promise<void>}
 */
export function clearDatabase(idbDatabase) {
  return new Promise((resolve, reject) => {
    const transaction = idbDatabase.transaction(
      idbDatabase.objectStoreNames,
      'readwrite'
    )
    transaction.addEventListener('error', reject)

    let count = 0
    for (const storeName of idbDatabase.objectStoreNames) {
      transaction
        .objectStore(storeName)
        .clear()
        .addEventListener('success', () => {
          count++
          if (count === idbDatabase.objectStoreNames.length) {
            // Cleared all object stores
            resolve()
          }
        })
    }
  })
}

Great ! Thank you so much for these helpful functions. I just can't figure out how to use import statement in the browser. How do I run the functions in chrome console for example ?

@shellking4 What do you refer to with "import statement"? Importing stuff from the idb-backup-and-restore.mjs file or calling the importFromJson function?

@shellking4 What do you refer to with "import statement"? Importing stuff from the idb-backup-and-restore.mjs file or calling the importFromJson function?

Yes you get it right. How can I do it in my browser console ?

No, I'm sorry, I don't get it right: What do you want to do in the console?
Do you mean
a) importing stuff from the idb-backup-and-restore.mjs or
b) calling the importFromJson function?

No, I'm sorry, I don't get it right: What do you want to do in the console?
Do you mean
a) importing stuff from the idb-backup-and-restore.mjs or
b) calling the importFromJson function?

well... I think wanna do both. The whole point is that I have data in indexedDB and in a few weeks I will wipe the system out because of some graphical problems. But I'd really like to backup the data I have in indexedDB and restore them later on. In my search for solutions I luckily landed here.

Okay, that should be relatively straightforward, under one condition: You'll still have the IndexedDB on the new machine, even though it may be empty. This is necessary because the export script from above can only export the DB's data, not its strucure.

For both exporting and importing data, you'll need an IndexedDB instance. Say you're wanting to back up the foo database, then you could get an instance in the console like this:

var db
var request = window.indexedDB.open('foo')
request.onerror = () => console.error('Could not open database')
request.onsuccess = () => { db = request.result }

Export:

1. Copy the exportToJson function (without the prepended export statement), paste it into the DevTools Console and hit Enter.
2. Run this: await exportToJson(db)
3. If you see some JSON output in the console it means that the export worked. Run copy($_) to copy the return value of the previous command (= the exported data) to your clipboard.
4. Save that exported data somewhere.

Import:

1. Copy the importFromJson function (without the prepended export statement), paste it into the DevTools Console and hit Enter.

2. Invoke it with the IndexedDB instance and the exported data as parameters: await importFromJson(db, exportedString).

   Note that you actually have to pass the exported data as a string, not as a JSON object. For example: If you exported and empty database {}, you'd have to pass into the function the following: await importFromJson(db, '{}').

---

This should be it. Probably test it before you actually rely on it. 🙂

thank you very much @loilo for the quick and clear answer. I followed the steps and did as you said. ran it since a while but no json output yet. I really don't know if it's working or not.

Mh, I cannot say from the distance what your problem might be, but I have created a Gist for you which includes the whole workflow for exporting a database. Copy the script as a whole and paste it into your browser console — that should clearly state whether it worked or not:

https://gist.github.com/loilo/ddfdb3c54fa474a89f71ce0660cd38b7

Mh, I cannot say from the distance what your problem might be, but I have created a Gist for you which includes the whole workflow for exporting a database. Copy the script as a whole and paste it into your browser console — that should clearly state whether it worked or not:

https://gist.github.com/loilo/ddfdb3c54fa474a89f71ce0660cd38b7

Thank you very much for helping me !
I pasted the gist in the console and hit enter then the console output '' Promise {} ''
After it says '' Uncaught RangeError: Invalid string length at JSON.stringify ().

I have up to 63 stores in the database. Would it be the cause ? Too much data ?

It seems like it, at least the error message is usually linked to that problem.

For these amounts of data you should probably back up your whole browser's user data. For Google Chrome, those are located at %userdata%\AppData\Local\Google\Chrome\User Data (Windows) respectively ~/Library/Application Support/Google/Chrome (macOS). Other browsers use very similar paths.

You should be able to back up that whole folder and restore it in the same place on a new machine. I have definitely done this successfully before, but, again, you better verifiy it to work with another PC or in a VM. (If I remember correctly, there was also the restriction that the new device needed to have the same user/password credentials to be able to read the backed up data on Windows — but I'm not sure on that. That might only affect stored passwords.)

It seems like it, at least the error message is usually linked with that problem.

For these amounts of data you should probably back up your whole browser's user data. For Google Chrome, those are located at %userdata%\AppData\Local\Google\Chrome\User Data (Windows) respectively ~/Library/Application Support/Google/Chrome (macOS). Other browsers use very similar paths.

You should be able to back up that whole folder and restore it in the same place on a new machine. I have definitely done this successfully before, but, again, you better verifiy it to work with another PC or in a VM. (If I remember correctly, there was also the restriction that the new device needed to have the same user/password credentials to be able to read the backed up data on Windows — but I'm not sure on that.)

Thank you soooooooooooooo much, you're a life saver
Thanks a lot 🙂😊.

It seems like it, at least the error message is usually linked with that problem.

For these amounts of data you should probably back up your whole browser's user data. For Google Chrome, those are located at %userdata%\AppData\Local\Google\Chrome\User Data (Windows) respectively ~/Library/Application Support/Google/Chrome (macOS). Other browsers use very similar paths.

You should be able to back up that whole folder and restore it in the same place on a new machine. I have definitely done this successfully before, but, again, you better verifiy it to work with another PC or in a VM. (If I remember correctly, there was also the restriction that the new device needed to have the same user/password credentials to be able to read the backed up data on Windows — but I'm not sure on that.)

Thank you soooooooooooooo much, you're a life saver
Thanks a lot 🙂😊.

Glad I could help. 🙂 Have a nice day! 👋

Glad I could help. 🙂 Have a nice day! 👋

Thank you ! Have a nice day too

Mh, I cannot say from the distance what your problem might be, but I have created a Gist for you which includes the whole workflow for exporting a database. Copy the script as a whole and paste it into your browser console — that should clearly state whether it worked or not:

https://gist.github.com/loilo/ddfdb3c54fa474a89f71ce0660cd38b7

Thanks very much for this script, it was just what I was looking for!

Thanks so much @loilo! In case anyone is wondering, the callback function that handles the cursor can also extract the key corresponding to each object in the store, if you want that information (as I did). The modification in exportToJson simply becomes:

allObjects.push([cursor.key, cursor.value])

and the corresponding modification in importFromJson within the loop:

 let [k,value] = toAdd ; 
 const request = transaction.objectStore(storeName).add(value,k)

Hope that helps someone and thanks again! Web tech is so dope , more info on cursors here

Okay, that should be relatively straightforward, under one condition: You'll still have the IndexedDB on the new machine, even though it may be empty. This is necessary because the export script from above can only export the DB's data, not its strucure.

For both exporting and importing data, you'll need an IndexedDB instance. Say you're wanting to back up the foo database, then you could get an instance in the console like this:

var db
var request = window.indexedDB.open('foo')
request.onerror = () => console.error('Could not open database')
request.onsuccess = () => { db = request.result }

Export:

1. Copy the exportToJson function (without the prepended export statement), paste it into the DevTools Console and hit Enter.
2. Run this: await exportToJson(db)
3. If you see some JSON output in the console it means that the export worked. Run copy($_) to copy the return value of the previous command (= the exported data) to your clipboard.
4. Save that exported data somewhere.

Import:

1. Copy the importFromJson function (without the prepended export statement), paste it into the DevTools Console and hit Enter.
2. Invoke it with the IndexedDB instance and the exported data as parameters: await importFromJson(db, exportedString).
   Note that you actually have to pass the exported data as a string, not as a JSON object. For example: If you exported and empty database {}, you'd have to pass into the function the following: await importFromJson(db, '{}').

This should be it. Probably test it before you actually rely on it. 🙂

Hi, Thanks for the script, But when I tried to dump the two emscripten_filesystem and gscyclone database on https://play.wo1wan.com/jjnext/play?id=540&mode=0&gstate=def, the script just copied {"FILES":[]} and then the page redirected to about:blank.

I also tried the script on https://gist.github.com/loilo/ddfdb3c54fa474a89f71ce0660cd38b7, however, it's the same problems.

I could not found any error in the console.

Am I correct that the importFromJson function requires that the target db has all the necessary stores created beforehand?

@moniuch Yes, if I remember correctly, this is the case.

@moniuch It simple to modify the code to create stores by JSON before the import.

I may have found a bug - I had a few object stores that were empty. When importing, because there were no records, delete importObject[storeName] was never run. I fixed it by adding a check before the for loop:

      const importObject = JSON.parse(json)
      for (const storeName of idbDatabase.objectStoreNames) {
        // ignore empty stores (we have to do this explicitly or the keys will never be deleted, and the promise will not resolve)
        if(importObject[storeName].length == 0){
            delete importObject[storeName];
            continue;
        }
        let count = 0
        for (const toAdd of importObject[storeName]) {
....