Tushar Sharma tusharf5

Reading Big Files in Node.js is a little tricky. Node.js is meant to deal with I/O tasks efficiently and not CPU intensive computations. It is still doable though but I'd prefer doing such tasks in languages like python, R etc. Reading, Parsing, Transforming and then Saving large data sets (I'm talking millions of records here) can be done in a lot of ways but only a few of those are efficient. Following snippet is able to parse millions of records without wasting a lot of CPU (15% - 30% max) and (40 MB - 60 MB max) memory. It is based on Streams.

The following program expects the input to be a csv file source eg. big-data.unpr.csv It saves the result as ndjson and not json as working with huge datasets is easier when done using ndjson format.

go to terminal in your project folder. a. run git config credential.helper store
run git pull
input your username and password
go back to sourceTree and run Fetch or Pull, it does not ask your password again.

OR

You need to update SourceTree config via "Preferences -> Git tab -> Git Version" by select "Use System Git", then everything works well

sudo pip install awscli # Installing aws cli
$ aws configure # Setup
$ aws s3 ls # You can see list of buckets

See credentials file for aws with cat ~/.aws/credentials

This is default profile

	/**
	* Retries a promise n no. of times before rejecting.
	*/
	async function retryPromise(promise, nthTry) {
	try {
	const res = await promise;
	return res;
	} catch (e) {
	if (nthTry === 1) {
	return Promise.reject(e);


	let obj = {
	id: '1',
	children: [
	{
	id: '2',
	children: [
	{
	id: '4',
	children: [

	// try here http://pdfmake.org/playground.html

	var dd = {
	content: [
	{
	columns: [
	{
	image:
	'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAMgAAABjCAYAAADeg0+zAAAACXBIWXMAABYlAAAWJQFJUiTwAAAQbUlEQVR42u1dh3tUVRbnf9hvv5WuJBAkhZKEJEAoZkICBKWpVAUERClSFQgl9CZIjYAiuAvLoq4FdEURRQQVFUGa9A5SpUsJ4ez9nXn35c3kvZk3aQQ49/t+32TevHLL+d1T7rkvZWrEPkECgcAeZaQTBAIhiEAgBBEIhCACgRBEIBCCCARCEIFACCIQCEEEAiGIQCAQgggEQhCBQAgiEAhBBAIhiEAgBBEIhCACgRBEIBCCCARCEOkIgUAIIhAIQQQCIYhAIAQRCIQgAoEQRCAQgggEQhCBQAgiEAiEIAKBEEQgEIIIBEIQgUAIIhAIQQQPOh6v08TVMSFIATuzuO7t9Cy35xXmOQVtZyjXBTq3IL/heEGeHxmXQlHxHh/g2P1IlDL3khi6s6rXbkzVajaiiFqNqJofIiyfOF93Pj7dDnoEX9/YdtDz6tCE6xCqYOrz8Il6oi3+z7F+Rvi1y7+t+notWG7r4v8M/34LRlzb61z2hXVc8D0sqgFVikigitXqMvA3jul2RcbdP0QpFRqkTr1mlNj4SYpLbmGLeAWcg/MfrZFEFVSnV41pyJ0daJbTv9Vt1JJiGzQPeF7NhKZch2ACFUhAcH2tpDRTyO0EEe1JUPWxayfqGF03lcKiG1DFCK9wgdhuiaJ/r9swgxJUXYD45AzXGqRuw5aW61pQjTrurkP9MB4YFxxLb9WFuvceQv0Gj2J06z2Y0p7qzP2Cc6rVbBgS+R9agkTFp1Dlx5NowdvL6Pr1v+jSpct09dp1W1y5cpX+vHiJtmzbQVN

	# bash/zsh completion support for core Git.
	#
	# Copyright (C) 2006,2007 Shawn O. Pearce <spearce@spearce.org>
	# Conceptually based on gitcompletion (http://gitweb.hawaga.org.uk/).
	# Distributed under the GNU General Public License, version 2.0.
	#
	# The contained completion routines provide support for completing:
	#
	# *) local and remote branch names
	# *) local and remote tag names

	# bash/zsh git prompt support
	#
	# Copyright (C) 2006,2007 Shawn O. Pearce <spearce@spearce.org>
	# Distributed under the GNU General Public License, version 2.0.
	#
	# This script allows you to see repository status in your prompt.
	#
	# To enable:
	#
	# 1) Copy this file to somewhere (e.g. ~/.git-prompt.sh).

	find . -name "node_modules" -type d
	find . -name "node_modules" -type d -exec rm -rf {} +

	#!/bin/sh

	echo "[Deploy] Building"
	yarn run build
	echo "[Deploy] Changing Directory"
	cd ./dist
	echo "[Deploy] Current Directory: ${PWD}"
	echo "[Deploy] Emptying existing bucket <bucket_name>"
	aws s3 rm s3://<bucket_name> --recursive
	echo "[Deploy] Uploading new files to <bucket_name>"