Skip to content

Instantly share code, notes, and snippets.

View fergiemcdowall's full-sized avatar

Fergus McDowall fergiemcdowall

  • Oslo
View GitHub Profile
@fergiemcdowall
fergiemcdowall / LevelDBFileReader.js
Last active January 10, 2022 14:56
A Gist to demonstrate writing and reading image files to LevelDB via the node module level up
var levelup = require('levelup')
var fs = require('fs');
var db = levelup('./imagedb')
db.put('name', fs.readFileSync('image.png'), { encoding: 'binary' }, function (err) {
db.get('name', { encoding: 'binary' }, function (err, value) {
fs.writeFile('image-copy.png', value, function (err) {
console.log('image-copy.png saved!');
});
})
@fergiemcdowall
fergiemcdowall / levelUpStressTest.js
Last active December 21, 2015 03:09
A gist that stress tests levelUP by inserting lots of batch files containing JSON. This can be configured to create a "memory leak" in levelUP.
//[email protected]
//A very simple stress test for levelUP
//make lots of LorumIpsum json files, put them into batches, and then
//cram as many of them as possible into LevelUP
//At the time of writing (15 aug 2013) this will create a memory leak
//adjust size of batches here
var totalBatchFiles = 300;
var totalDocsPerBatchFile = 100;
@fergiemcdowall
fergiemcdowall / gist:6775562
Created October 1, 2013 08:43
lev install errors
C:\Users\fmcdowall>npm install lev -g
npm http GET https://registry.npmjs.org/lev
npm http 304 https://registry.npmjs.org/lev
npm http GET https://registry.npmjs.org/level
npm http GET https://registry.npmjs.org/optimist/0.3.4
npm http GET https://registry.npmjs.org/deepmerge
npm http GET https://registry.npmjs.org/level-delete-range/0.1.0
npm http GET https://registry.npmjs.org/tabulate/0.0.1
npm http GET https://registry.npmjs.org/event-stream
npm http GET https://registry.npmjs.org/reconnect
@fergiemcdowall
fergiemcdowall / backup.json
Last active August 29, 2015 14:05
How to pipe a file of key-value pairs into levelUP
{"key":"john","value":"lennon"}
{"key":"paul","value":"mccartney"}
{"key":"ringo","value":"starr"}
{"key":"george","value":"harrison"}
@fergiemcdowall
fergiemcdowall / gist:a20a1b67811611f4c6cc
Created May 2, 2015 11:55
Make a snapshot of a search-index
var si = require('search-index')({logLevel:false});
var fs = require('fs');
si.snapShot(function(rs) {
rs.pipe(fs.createWriteStream('backup.gz'))
.on('close', function() {
console.log('great success dear leader')
})
.on('error', function() {
console.log('err')
@fergiemcdowall
fergiemcdowall / gist:dabf0e943de6e02a9f86
Created May 2, 2015 14:02
replicate from search-index snapshot
var si = require('search-index')({logLevel:false});
var fs = require('fs');
si.replicate(fs.createReadStream('backup1.gz'), function(msg){
console.log('restored');
});
@fergiemcdowall
fergiemcdowall / norchifier.js
Created May 12, 2015 14:14
a program that norchifies a world-bank-dataset
var fs = require('fs');
var _ = require('lodash');
var data = JSON.parse(fs.readFileSync('world-bank-projects.json'));
var padInt = function(intIn) {
return ("000000000000000" + intIn).slice(-15);
}
var processDoc = function(datum) {
node_modules
sample_index
@fergiemcdowall
fergiemcdowall / README.md
Last active March 8, 2025 10:51
How to use pandoc to generate HTML from GitHub-flavoured markdown

Here is a little recipe for generating html from github-flavoured markdown. CSS extracted and modified from https://github.com/sindresorhus/github-markdown-css on 22-05-2019. At some point this CSS will probably have to be updated, but for now it works.

  1. Install pandoc 2.x or higher

  2. Save github-markdown.css (below) to ´~/.pandoc/github-markdown.css´

  3. To convert a file called README.md (change as appropriate) run ´pandoc --standalone -c ~/.pandoc/github-markdown.css -f gfm -t html README.md´

@fergiemcdowall
fergiemcdowall / cars.js
Last active October 14, 2019 07:45
Generate a randomised JSON array of cars
// randomise function
const random = (...list) => list[Math.floor(Math.random() * list.length)]
// create random numbers within a range
const getRandomInt = (min, max) => {
min = Math.ceil(min);
max = Math.floor(max);
return Math.floor(Math.random() * (max - min + 1)) + min;
}