Skip to content

Instantly share code, notes, and snippets.

@FirstVertex
Created April 27, 2020 13:27
Show Gist options
  • Save FirstVertex/333a2bc2b7fcd914ed87421a4a9c61a9 to your computer and use it in GitHub Desktop.
Save FirstVertex/333a2bc2b7fcd914ed87421a4a9c61a9 to your computer and use it in GitHub Desktop.
Create a stream from an ArrayBuffer
import * as Stream from 'stream';
class ArrayBufferStream extends Stream.Readable {
constructor(private _arrayBuffer: ArrayBuffer, opts?: Stream.ReadableOptions) {
super(opts);
}
private _chunk_size = 1024 * 16;
private _pointer = 0;
_read(size: number) {
size = size || this._chunk_size;
if (this._pointer + size >= this._arrayBuffer.byteLength) {
size = this._arrayBuffer.byteLength - this._pointer;
}
const from = this._pointer;
this._pointer += size;
// we use setTimeout here to yield to the UI thread, prevent it from locking up
setTimeout(() => {
if (size) {
this.push(Buffer.from(this._arrayBuffer, from, size));
} else {
this.push(null);
}
}, 0);
}
}
@FirstVertex
Copy link
Author

I use this with node-csvtojson to efficiently stream large csv files (500MB / 2,000,000 records) into ag-grid.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment