Skip to content

Instantly share code, notes, and snippets.

View piscisaureus's full-sized avatar
🦕

Bert Belder piscisaureus

🦕
View GitHub Profile
@piscisaureus
piscisaureus / gist:7878977
Created December 9, 2013 19:12
OS X threaded pwrite() bug
The issue:
Concurrent pwrite() calls from different threads cause file corruption
on OS X, even if the file regions that are being written to don't
overlap.
We're not sure whether this may have been fixed in more recent versions
of OS X. So far we've been unable to reach someone within apple that
was able to provide more information or could fix it.
@piscisaureus
piscisaureus / domains2.md
Last active December 28, 2015 06:09
Domain spec

Basic tasks

When you are in a Domain and you spawn off an async call (like fs.stat), the callhack for that async call gets executed in the domain. With tasks this is the same, so far, nothing new.

However, there is no 'end' to a domain. At some point the Domain object might become unreachable and gets GC'ed. That's all.

Think of a Task as an extension to a domain, namely a domain that has a callback. That callback is called - automatically - in the context of it's parent domain, whenever, by the end of the current tick, we know that the domain will never be entered again.

So the basic strategy to make this work.

var http = require('http');
var cache = {};
function curl(url, cb) {
if (cache[url])
// WRONG: synchronous callback
return cb(null, cache[url]);
var data = '';
task.create(function ConcatTwoFilesAndThenSomeTask() {
fs.readFile('some/template', 'utf8', function(err, file1) {
if (err) throw err;
fs.readFile('something/else', 'utf8', function(err, file2) {
if (err) throw err;
var data = file1 + '--' + file2;
data = data.replace(/foo/g, 'bar');
return data;

Disclosure: I am one of the StrongLoop founders.

If John Lennon was born half a century later he'd probably be famous for the following words:

"Life is What Happens On The Internet While You’re Busy Making Other Plans"

Something was said on the Internet that warrants a response.

Let me paraphrase a conversation I have a couple of times at every conference I attend. It has happened at Nodeconf and Nodeconf.eu this year, and I am certain it will happen again next week at lxjs.

var spawnSync = require('child_process').spawnSync;
var stdio = [
{ type: 'pipe', readable: true, input: new Buffer('Fubar!') },
{ type: 'pipe', writable: true },
{ type: 'pipe', writable: true },
{ type: 'inherit', fd: 0 }
];
var options = {
#define JS_STRING_MAP(XX) \
XX( address ) \
XX( AF_INET ) \
XX( AF_INET6 ) \
XX( AF_UNSPEC ) \
XX( args ) \
XX( argv ) \
XX( atime ) \
XX( blksize ) \
Thread 23 (process 18336):
#0 0x00007fff8c2cb192 in __workq_kernreturn ()
#1 0x00007fff8d384594 in _pthread_wqthread ()
#2 0x00007fff8d385b85 in start_wqthread ()
Thread 21 (process 18336):
#0 0x00007fff8c2cb192 in __workq_kernreturn ()
#1 0x00007fff8d384594 in _pthread_wqthread ()
#2 0x00007fff8d385b85 in start_wqthread ()
@piscisaureus
piscisaureus / write-behind.js
Last active December 20, 2015 00:38
Write behind transform stream
module.exports = WriteBehind;
var assert = require('assert'),
Transform = require('stream').Transform,
inherits = require('util').inherits;
var DEFAULT_SIZE = 1 * 1024 * 1024; // 1 mb
var WriteStream = require('fs').WriteStream;
var BUF_SIZE = 1 * 1024 * 1024;
var CHUNK_MAX = 100 * 1024;
WriteStream.prototype.__write = WriteStream.prototype.write;
WriteStream.prototype.__end = WriteStream.prototype.end;
WriteStream.prototype.__flush = function() {
if (this.__writeBehindBuffer &&