Skip to content

Instantly share code, notes, and snippets.

@dominictarr
Last active June 30, 2024 05:21
Show Gist options
  • Save dominictarr/4832db6625112016c8c1 to your computer and use it in GitHub Desktop.
Save dominictarr/4832db6625112016c8c1 to your computer and use it in GitHub Desktop.

here are the most useful pull streams modules

combining pull streams

a library of simple functions that will be familiar functional programmers.

turn an array of source streams into a single stream. One stream is read, then the next, then the next.

create a place holder when you don't have the actual stream just yet. eg.

read from many source streams at once. (contrast this with pull-cat)

merge ordered sources together. compares the next item of each source and outputs the lowest one.

run an async operation for each item in the stream, in parallel, but the output order is the same as the input was. you can set how many concurrent operations are allowed.

a source stream that you can push items onto as a queue.

create many source streams and add another element to them with a notify function. useful for creating apis with realtime streams.

create a source and sink stream that are connected. This is sometimes useful for testing duplex streams.

a pull stream version of through.

traverse nested tree structures in either depth first, width first, or leaf first order.

process windows of data in some way. real time aggregations of time slices of realtime input etc.

file system and databases

recursively search for files on your fs, supports your familiar globs, plus ** (recursive) and ... (recursive towards the root)

read and write to leveldb via pull streams, supports streams of realtime changes!

working with text streams

split a incoming binary or text stream into lines (or by a given marker)

output a stream as valid json. line delimited json is also supported.

working with binary streams

control exactly how many bytes to read and write in a binary duplex stream.

Transform a stream of buffers so that the incoming buffers are split into random overlapping sizes. Useful for testing binary stream parsers that should be able to handle message boundries in different places

read a certain number of bytes ahead in a binary stream.

working with networks

pull stream wrapper for websockets.

combines rpc with stream multiplexing.

integration with node streams

wrap a node stream in pull streams and vice versa.

cryptography

encrypt a one way stream

secure and private duplex connection

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment