node and npm - Personal notes, 2018

Personal Notes on some Nodejs related courses, npm, and Modern Javascript

"Node.js: Getting Started"; "Advanced Node.js" by Samer Buna;

"NPM Playbook" by Joe Eames;

and other courses and tutorials

Notes

Node is based on v8 javascript engine (which has itself no concept of I/O built in).

As a webserver, Node was designed to serve many small files to many connected clients in parallel. Node also excels as a command line tool for scripting (replacing bash for certain tasks), and platform for client-side apps (e.g. as electron).

helloworld.js

#!/usr/bin/env node
console.log("Hello World")

process.stdout.write("Hello stdout\n")

var name = process.argv[2];
console.log("Hello " + name);

node hellowrld.js Knut outputs

Hello World
Hello stdout
Hello Knut

console Object is implemented on top of C code, wrapper for object process.stdout.
When node is used as CLI tool, concurrency aspects are less common, and it is generally okay to use synchronous API calls like fs.readFileSync()

The node REPL

For getting the most out of the REPL, install rlwrap commandline utility on linux (install with package manager, e.g. apt).

rlwrap:

allows the editing of keyboard input for any other command. Input history is remembered across invocations, separately for each command; history completion and search work as in bash and completion word lists can be specified on the command line

Enables ctrl-r reverse search in node repl.

NODE_NO_READLINE=1 rlwrap node

Type .editor to enter multiline+multistatement mode. Type .break to get out of this mode Type .help to see list of commands.

.load/.save useful for storing/saving the commands edited in the session: .load ./mycommands.js (small files)

(use let j = require("./src/data.json"); Object.values(j) to load larger structures, e.g. results from API calls)

Type c double-tab to get autocompletion suggestions for kewords beginning with c. Type v8 tab-tab to see v8 commands

Type double-tab on a single line. Receive list of Global Javascript Objects, Built-in Noddde global Objects (Uppercase first Char), and built-in node modules (lowercase).

_ stores the value of the last evaluated expression (like $_ in Perl). Only available in repl.

Buffer.allocUnsafe(900).toString() - dumps a bunch of memory. (Fun: check if there's something recognizable in there)

require.extensions

require() - looks for *.js files, then for *.json files (data,objects), then for *.node binaries (c++ addons. need gyp mdoule to compile as addon)

In setTimeout(func, delay, arg1, arg2, arg3, ...) we can use for func: func(arg1, arg2, arg3, ...)

Is it run as a script on the command line node myscript.js arg1 arg2, or being required?

if(require.main == module){
// run as a script: No
console.log("This file must be required(), cannot run as a script");
process.exit(1);
} else {
// Being required
  module.exports = print;
}

`let` vs `var`

var Keyword: for(var i ...) loop : variable i still accessible outside loop let i - variable is then undefined

const keyword: Cannot reassign. to a variable. Consequence: scalar assignments are immutable. However, nested objects: elements at deeper levels such objects are still mutable.

use Object.freeze() - for 1 level immutability
use Immutable.js package - for deep-copy immutability

Module 2 - "Modern Javascript" is well worth reading

NPM

npm -l (long help)    show full usage sho
npm help-search prune

npm set init-author-name 'Knut Behrends' // writes default to /home/knut/.npmrc

npm list --depth 0 --global --long true  (see description of packages in node_modules)

npm update -g eslint  #// recommended: update one library at a time

npm install <exact github URL> // install a fork of a module from github
npm i expressjs/express#4.14.3 install a specific version from  github
npm i --json -g --dry-run gyp  // what would be installed


npm install gist:8459... #// install from a github gist: use hashvalue
npm install ../my/fs-path  --save #// install from a folder, avoid err with --save

npm prune --production //quite useful for bringing a module into a proction setting

npm home       // open homepage
npm repo #// open webpage with original  repo (e.g. on github)

npm shrinkwarp   // fix version numbers (= operator) in package.json

npx eslint #// uses local eslint package binary

npm repos on github must use tags and be tagged as "release" then use npm publish

npm Registry:

Humanreadable webpage: https://www.npmjs.com/package/lodash Package info as JSON: http://registry.npmjs.org/lodash

npmjs account: knbknb00 C.-

npm help npm-scripts
npm outdated
npm update lodash@latest

Module 6 -Modules and Concurrency

all module files in node are (invisibly) wrapped inside a function: require('module').wrapper

[ '(function (exports, require, module, __filename, __dirname) { ',  '\n});' ]

this enables scoping.

require.resolve() // checks a .js file but does not execute it . Use for optional dependencies

require() loads only once (stops if found on the "PATH", loads partially if circular reference is found)

default filename is index.js

require('something') 
// see 
require.extensions
{ '.js': [Function], '.json': [Function], '.node': [Function] }
// resolves first to require.js, but can also load `something.json`, something.node
//better to be explicit then :
require('something.json')

// try
require.extensions[".json"].toString()

export with module.exports = { ... }

Module 7 - working with web servers

Use nodemon instead of node for hot-reloading while doing web development. Install it globally. Use it as an optional package inside package.json.

req object is of tpe/class http.IncomingMessage

before running npm install, un npm init -y to create a package.json file which documents what has been installed. npm is smart in finds stuff from node_modules/ dir and puts it into dependencies sub-object.

node

Useful options/flags

node -c <myscript.js>  # compile/check
node -e #eval 
node --use-strict
node -p "process.versions" # eval and print 

# for diagnosis and capability checks
node --v8-options | grep -v "type:" | wc -l  # node/v8 have 424 
node --v8-options | grep -v "in progress\|type:" | grep harmony # list all  harmony flags (n ~ 18)
node --harmony ## enable all completed harmony features ("late beta status")
node --harmony-shipping  # (enable all shipped harmony features)
node --v8-options | grep "in progress"  ### list all flags, filter flags in beta (n ~ 5)

Use StringDecoder when working with Buffers

executing callbacks with setImmediate() can take precedence over 0-millisecond setTimeout(). another API, process.nextTick(cb, err) takes even higher precedence.

Callbacks, Promises and Async/Await

"Official" method to work with async code (for historical reasons): callbacks-style.

Follow callback calling convention: function(err, cb) {} // error first arg is nullable

Alternative: Promise Objects:

can handle success and failure separately
can chain async functions instead of nesting them

More improvements:

Function generators.
async/await

calling asyncronously inside loops is much easier with these language constructs

Event Emitters

this.emit('eventname', data, arg2, [...]) // can put many arguments here

there is a special error event:

myListener.on('error', console.error)

Builtin Libraries

os.constants.signals

fs module const fs = require('fs')

all methods to inteact with filesystem objects exist in syncronous and async forms, eg.

fs.readFile() and fs.readFileSync()

fs.readdir and fs.readdirsync

var stream = fs.createReadStream() + stream.on("data", function(chunk){}) - e.g. for large files, output flushed immediately

stream.pipe(fs.createWriteStream()) - streams are pipeable, devil is in the details (which stream causes which event?)

methods handle exceptions differently.

fs.stat() method useful

path module const path = require('path')

util module

util.inspect(myobject, {depth: 0}) returns onl high lvel view of an object, as String

on console: console.dir(global, {depth: 0})

assert module - can be used for testing. Simple but usable. assert.ifError(somthingTruthy)

net module

const server = require('net').createServer(); - then work with sockets

server has class http.IncomingMessage,

http and https modules

http.get() methods work exactly the same for both modules

https: create selfsigned certificate with openssl req -x509 -newkey rsa:4096 -keyout myrsakey.pem -cert myrsacert.pem -nodes

const http = require('http'); const req = http.request({}); // or http.get()

http.Server,

http.Agent, - req.agent http.ClientRequest - req, returned by http.request() or http.get(), http.ServerResponse

url module

try the url.parse("http://..", true) and url.format() methods. true arg makes method also parse Querystring. can also use querystring.parse() method. querystring.stringify({myobj}) does the inverse

debugger

call : node debug myscript.js

add breakpoint before first debugger command with sb(2) - on line 2 the do repl command

to use the chrome devtools debugger,

use node --inspect-brk myscript.js,
or node --inspect-brk node_modules/.bin/webpack --env production.
open Chrome devtools, click on green node icon (dedicated node debug devtools), step through the code.
or open url that node debugger shows: , e.g. ws://127.0.0.1:9229/3c61ec39-5cd7-4731-ad3e-0cf9bf3242a5

(node uses SIGUSR1 for its debugger. You cannot use it, use SIGUSR2 in your JS programs instead, if need be.)

another alternative npm install debug, in bash terminal set env var DEBUG=* node index.js => lots of useful output, e.g. for express apps

in index.js set app 'namespace': var debug = require('debug')('app'); in bash terminal set env var DEBUG=app node index.js => less output, only relevant output

npm install --save-dev morgan - log web traffic data to console

in index.js: app.use(morgan('combined')); // logs http requests to console. Can also use 'tiny' arg.

scalability

for better workload parallelism and better robustness

cloning (multiple versions of same thing), decomposing (microservices), splitting (sharding / data parallelism)

child_process module

exec() execFile(), fork(), spawn() - (also their ...Sync() equivalents)

spawn() - does not create a shell => efficient. Also better when bigger dataset returned, will be 'output-autoflushed'

exec() - can use shell pipes / command syntax directly in ''. Ok if data returned is small (node buffers all)

spawn('find . -type f', {stdio: 'inherit', shell: true, cwd: {/tmp}, env: {A: 2}) - issue shellpipe directly, but output autoflushed

child events: disconnect, message, error, close (closed the stream), exit (does not close the stream)

child.stdin (writable), child.stdout, child.stderr (readable)

fork() - for bidirectional communication based on message events, used for load balancing with cluster module

tooling

npm i forever - restarts node if it crashes

npm i -S debug nodemon

in package.json: scipts.start: "DEBUG=app nodemon app.js"

nodemon: use this config in package.json:

npm i node-static - can serve static files (can sniff MIME type, set header, gzip...) without much configuration

  "nodemonConfig":{
    "restartable":"rs",
    "ignore":["node_modules/**/node_modules"],
    "delay": "2500", //ms
    "env": {
      "NODE_ENV": "development", 
      "PORT": 4000 // use with process.env.PORT
    }
  }

knbknb/node-npm-notes.md

Personal Notes on some Nodejs related courses, npm, and Modern Javascript

Notes

`let` vs `var`

NPM

npm Registry:

node

Callbacks, Promises and Async/Await

Event Emitters

Builtin Libraries

debugger

scalability

tooling

See also

knbknb/node-npm-notes.md

Personal Notes on some Nodejs related courses, npm, and Modern Javascript

Notes

let vs var

NPM

npm Registry:

node

Callbacks, Promises and Async/Await

Event Emitters

Builtin Libraries

debugger

scalability

tooling

See also

`let` vs `var`