tooling.md

Diagnostic tooling is important to the consumers of Node.js. It is used both in development and in production in order to investigate problems. The failure of one of these tools may be as big a problem for an end user as a bug within the runtime itself.

The Node.js project has assessed the tools and their relative importance and put them into one of the following tiers.

• Tier 1 - Tool must always be working for all Current and LTS Node.js releases. A release will not be shipped if the test suite for the tool is not green. For a tool to be considered for inclusion in this tier it must have a good test suite and that test suite and a job must exist in the Node.js CI so that it can be run as part of the release process. No commit to master should break this tool if the next release is within 1 month.

• Tier 2 - Must be working for all LTS releases. An LTS release will not be shipped if the test suite for the tool is not green. For a tool to be considered for inclusion in this tier it must have a good test suite and that test suite and a job must exist in the Node.js CI so that it can be run as part of the release process.

• Tier 3 - If possible its test suite will be run at least nightly in the Node.js CI and issues opened for failures. Does not block shipping a release.

• Tier 4 - Does not block shipping a release.

• Unclassified - tool is new or does not have the required testing in the Node.js CI in order to qualify for a higher tier.

The choice of which tier a particular tool will be assigned to, will be a collaborative decision between Diagnostics WG and Release WG. Some of the criteria considered might be:

• The tool fits into a key cateogry as listed below
• Whether the tool is actively used by the community
• Availability of alternatives
• Impact to the overall ecosystem if the tool is not working
• Availability of reliable test suite that can be integrated into our CI
• Availability of maintainer or community collaborator who will help resolve issues when there are CI failures.
• The tool is maintained the Node.js foundation Github org

The current categories of tools that fall under these Tiers are:

• FDDC
• Tracing fill this is

The following tools are currently assigned to Tiers as follows

Tier 1

Tier 2

Tier 3

Tier 4

Not yet classified

Tool Type	Tool Name	Regular Testing In Node.js CI	Integrated with Node.js	Target Tier
FFDC	node-report	No	No	1
Heap/mem	llnode	?	No	2
Heap/mem	mdb_v8	No	No	4
Heap/mem	node-heapdum	No	No	2
Profiler	V8 cpu profiler	No	Yes	1
Profiler	V8 heap profiler	No	Yes	1
Profiler	V8 sampling heap profiler	No	Yes	1
Tracing	trace_events	No	Yes	1
APM	Async Hooks	?	Yes	1
Debugger	V8 Debug protocol	No	Yes	1
Debugger	Command line Debug Client	?	Yes	1
Debugger	Chrome Dev tools	?	No	3
Tracing	DTrace	No	Partial	3
Tracing	LTTng	No	Removed ?	N/A
Tracing	ETW	No	Partial	3
Profiling	Linux perf	No	Partial	?
Profiling	Windows xperf	No	?	?
Profiling	Ox	No	No	to early
Profiling	node-clinic	No	No	to early
Debugger	Chakracore -time-travel	No	data source only	to early

????

• SystemTap - nothing specific to Node?
• eBPF tracing tool - anything specific to Node (nodegc and nodestat, anything else)?

See issue: nodejs/diagnostics#157

Awesome! I think that's exactly what we need.

Just a few comments:

• DTrace could also be classified as a Profiling tool
• If "Regular Testing In Node.js CI" also includes our V8 CI, then I believe we can change Linux perf to "Yes" in that column
• Systemtap situation is similar to DTrace (for tracing): we have a file specifying tracepoint in Node.js core (). I would add it as Tracing | Systemtap | No | Partial | ?
• eBPF has tools for tracing and profiling which could be used instead of Systemtap and Linux perf. There are also some helper tools to easily trace Garbage Collection and other events (nodegc and nodestat).

Does Tool Type APM refer to the products from e.g. Elastic, New Relic, AppDynamics, Dynatrace,...?

If yes I would not bind them to Tool Name "Async Hooks" as above products do much more than context tracing using AsyncHooks.

Actually "Async Hooks" is not a tool it's a core functionality used (and needed) by APM tools. But also other tools like node-clinic or bubbleprof and as far as I know even node core in Domains use and need it.