Skip to content

Instantly share code, notes, and snippets.

View pulkitsinghal's full-sized avatar

Pulkit Singhal pulkitsinghal

View GitHub Profile
var _ = require('underscore');
var omitEmptyValues = function omitEmptyValues(data) {
if(!_.isObject(data)) {
return data;
}
else {
if (_.isArray(data)) {
var clone = [];
_.each(data, function(value) {
@pulkitsinghal
pulkitsinghal / dropbox
Created July 19, 2017 06:22 — forked from thisismitch/dropbox
/etc/init.d/dropbox
#!/bin/sh
### BEGIN INIT INFO
# Provides: dropbox
# Required-Start: $local_fs $remote_fs $network $syslog $named
# Required-Stop: $local_fs $remote_fs $network $syslog $named
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# X-Interactive: false
# Short-Description: dropbox service
### END INIT INFO
@pulkitsinghal
pulkitsinghal / avoidLeaksWithOneJobPerWorkerRun.js
Created June 3, 2017 18:19
Proof: one job per worker run
const os = require('os');
var kue = require('kue');
var queue = kue.createQueue({
redis: process.env.REDIS_URL
});
var JOB_NAME = 'crawl';
@pulkitsinghal
pulkitsinghal / README.md
Last active May 13, 2024 09:40
How can a nodejs process running inside a docker container, get that container's id?

Problem

How can a nodejs process running inside a docker container, get that container's id?

Solution

the hostname seems to be the short container id in Docker v1.12

It works and this idea was found in this SO post.

@pulkitsinghal
pulkitsinghal / worker.js
Created June 2, 2017 21:51
KUE - sure fire way to shutdown after processing only one payload
var kue = require('kue')
var queue = kue.createQueue({
redis: process.env.REDIS_URL
});
var JOB_NAME = 'crawl';
queue.process(JOB_NAME, function(job, ctx, done){
console.log((new Date()).toLocaleString(), 'inside queue.process("'+JOB_NAME+'") for job id:', job.id);
@pulkitsinghal
pulkitsinghal / worker.js
Created June 2, 2017 21:45
KUE - experiment w/ various ways to shutdown after processing only one payload
var kue = require('kue')
var queue = kue.createQueue({
redis: process.env.REDIS_URL
});
queue.process('crawl', function(job, ctx, done){
console.log((new Date()).toLocaleString(), 'inside queue.process("crawl") for job id:', job.id);
doSomething(job.data, ctx, done);
//exit(); // Scenario C: don't like it because shutdown can happen before the job finishes,
@pulkitsinghal
pulkitsinghal / README.md
Last active May 31, 2017 23:54
Setup VSCode (Visual Studio) with SFTP - syncing allows you to use IDE locally with files on powerful remote machines
@pulkitsinghal
pulkitsinghal / sample.js
Created November 8, 2016 02:44
Use callbacks to fetch from AWS S3, on a machine with pre-configured access
var AWS = require('aws-sdk');
var s3 = new AWS.S3();
s3.getObject({
Bucket: 'bucket-name',
Key: 'file-name.ext'
}, function (err, data) {
if (err) {
logger.error(err);
returnResponse(res, 500, logger, 'Internal Server Error') ;
}
@pulkitsinghal
pulkitsinghal / README.md
Last active October 21, 2016 20:20
Wrap sockets for use with angular - part1
  1. Install a 3rd party utility to help with wrapping the socket library:
bower install angular-socket-io`
  1. Update index.html to provide a websocket implementation like socket.io and the wrapper utility:
<script src="//cdn.socket.io/socket.io-1.1.0.js"></script>
<script src="bower_components/angular-socket-io/socket.min.js"></script>
@pulkitsinghal
pulkitsinghal / notifier.js
Last active October 16, 2016 18:20
Worker > Redis > Notifier Service (multiple nodes) > Browser
// starting point was: https://github.com/mminer/blog-code/blob/master/pattern-for-async-task-queue-results/notifier.js
var express = require('express'),
app = express(),
server = require('http').Server(app),
io = require('socket.io')(server),
bodyParser = require('body-parser');
// After scaling up, multiple servers can only work with sockets connected to them.
// The socket.io-redis package can solve this by making redis the datastore instead of memory.