😏

Budleigh Salterton MichalCz

😏

The founder of the company @scramjetorg, developer of the Scramjet Framework, geek and an unserious cyclist...

MichalCz / simple-rest-to-db.js

Created April 6, 2017 15:40

How to crawl a rest API and save response to DB with Scramjet.


	const scramjet = require("scramjet");
	const stream = require("stream");
	const request = require("request-promise");
	const config = require(process.env.CONFIG);

	let cnt = 0;
	// build a stream of indexes
	new scramjet.DataStream({
	read() {

MichalCz / scramjet.js

Last active April 8, 2017 11:56 — forked from kharandziuk/scramjet.js

	const scramjet = require("scramjet");
	const {getAPI,insertDB} = require('./io-simulators');

	const ROWS = 1000;

	class APIReadable extends scramjet.DataStream {
	constructor(options) {
	super(options);
	this.offset = 0;
	}

MichalCz / travis-local-build.log

Created October 3, 2018 14:07

	michal@miura:/ccc/apps/worker-4.2.0⟫ docker run -it travisci/ci-nodejs:packer-1494866191 /bin/bash
	root@3bb11d58773a:/# su -l travis
	travis@3bb11d58773a:~$ git clone --depth=50 --branch=master https://github.com/signicode/scramjet-core.git signicode/scramjet-core
	Cloning into 'signicode/scramjet-core'...
	remote: Enumerating objects: 816, done.
	remote: Counting objects: 100% (816/816), done.
	remote: Compressing objects: 100% (311/311), done.
	remote: Total 816 (delta 549), reused 735 (delta 494), pack-reused 0
	Receiving objects: 100% (816/816), 245.59 KiB \| 0 bytes/s, done.
	Resolving deltas: 100% (549/549), done.

MichalCz / remote-code.sh

Created October 16, 2019 11:30