DuckDB cluster mode issues

README.md

This gist showcases how DuckDB on nodejs fails silently to handle the same read only database on multiple nodejs processes when running in cluster mode. This behaviour is consistent with the duckdb CLI interface.

Calling (using an in :memory: database)

node cluster.js

outputs the correct result

Forked child
Forked child
Forked child
Forked child
Child done
Child done
Child done
Child done
42
42
42
42

If we use a file database

node cluster.js some-db.duckdb

it'll only run the query once, all other processes in the cluster will fail silently

Forked child
Forked child
Forked child
Forked child
Child done
Child done
Child done
Child done
42

cluster.js

const cluster = require('cluster');

if (cluster.isMaster) {
  // Fork workers.
  for (var i = 0; i < 4; i++) {
    cluster.fork();
  }
  cluster.on('error', function(err) {
    console.error(err);
  });
} else {
  console.log('Forked child');
  require('./index')(process.argv[2] || ':memory:')
  console.log('Child done')
}

index.js

const duckdb = require('duckdb')

module.exports = (dbPath = ':memory:') => {
  const db = new duckdb.Database(dbPath);
  db.all('SELECT 42 AS fortytwo', function(err, res) {
    if (err) {
      throw err;
    }
    console.log(res[0].fortytwo)
  });
}

if (require.main === module) {
  module.exports(process.argv[2] || ':memory:')
}

package.json

{
  "name": "duckdbcluster",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "MIT",
  "dependencies": {
    "duckdb": "^0.3.0"
  }
}

I'm having the same issue and found this Gist. Were you able to solve this issue? When I upgrade to [email protected], I get the following error when using a file database:

[Error: Connection Error: Connection was never established or has been closed already] {
  errno: -1,
  code: 'DUCKDB_NODEJS_ERROR',
  errorType: 'Connection'
}

The issue is that there cannot be multiple processes and WRITE mode, so setting duckdb to 'READ_ONLY' will fix this issue.

Gist with update: https://gist.github.com/ataft/42b7cc3d4055cf6b5a62a5f1f57da4b2