This document outlines how V2.0 n8n nodes are developed to ensure:
- Code Clarity
- Code Reduction / Standardization
- Error Handling @ Scale It outlines a new improved format for node development that will help n8n scale to millions of APIs without continuous maintenance but will also provide developers new to n8n a way to build nodes easily with standardized helper functions.
All of n8n node development in respect to:
- Resources (How to more efficiently group resources under a node)
- Operations (How to reduce code duplication across operations)
- Errors (How errors are parsed from APIs / consistently displayed)
Node: An n8n abstraction of an API that is the essential building block of the n8n system. Resources: A set of operations under an API Section Operations An API command execution API Error Object The Error Response Object provided from an API API Error Transformer The API Specific Transformer to convert an API's Error Object into n8n's internal Error Object Structure. JSDOC A way to document code in JavaScript | https://jsdoc.app/
N / A
Below is given both theoretical implementations and problems with code example solutions for real-life implementation on the n8n system.
Since V1.0 has NodeMaker, we will want this new architecture to be able to utilize that software to ensure that node development can be auto-generated as much as possible. The main point here is that we want consistency and high-quality code to be in EVERY node, not just nodes written by professionals. This means that n8n as usual as the "senior" team with the most contextual knowledge should have the most responsibility in terms of structure. Thus, we will be enforcing best practices that are flexible enough to handle any API integration but strict enough to ensure the highest code quality possible. In V2.0, our standardized structure also helps us write CODMODs which we can use to programmatically migrate thousands of API nodes at once without failure or mistakes as a human engineer would.
Previously, in V1.0 nodes error handling is handled at a node's individual level. This includes inconsistent Error Messaging and lack of Trace Logs. At a scale of 1,000 nodes, this becomes unmaintainable as with new PRs being issued day by day or changes needing to be made at an n8n level for error representation is currently unmaintainable and quite frankly, impossible. With the V2.0 implementation below n8n will take control of both the formatting of the Error Messaging as well as the information it chooses to display. As an added benefit, we will also provide an Error Object Dump (Raw View) so that the workflow user can clearly debug solutions in which APIs do not conform to international API standards (Which as we continue to add new APIs will become a reality).
Previously, in V1.0, errors are essentially dynamic strings. Due to the nature of dynamic strings, you would not be able to effectively and consistently catch errors across a large of set of nodes. With V2.0 errors are standardized across the n8n Standardization Error Object and layered with Standard HTTP Codes. This ensures that nodes can now ACCEPT errors as conditional actions as shown in the business case below.
In V1.0, it's very difficult to understand what functionality is supported by each node through the documentation. Currently, this is being manually done and is not scalable or maintainable as APIs change. In V2.0, our goal is to have code that is clearly readable to quickly parse what requests the API can form but also be parsable by a document generation tool. JSDoc is currently the leading standard but may not apply to our use case where a custom documentation generator would require less overhead in terms of documentation. Further clarification on how much documentation is required will bring visibility to this issue. The auto-generation we would provide would link the documentation to the code directly which is the main advantage of V2.0.
In V1.0, Nodes had inconsistent resources/operation structures that led to issues such as
- Difficult to use other nodes as model nodes (Nodes that can be referenced when creating new nodes)
- Inconsistent & Inefficient HTTP Request creations for groups of operations
- A bad developer experience since resources/operations did not have auto-complete or typescript protection. In V2.0, with a standardized, modular, and efficient structure for defining resources and operations, we are able to solve all the above problems while maintaining a large level of flexibility for APIs that are very custom as well the ability to self-document as we explain below.
2.1. Raw View React-UI Component This UI component is the existing one n8n is currently using in V1.0 for Error Messages Display, but we will be modifying it to include a button with the text RAW Error Message to showcase to the user the ability to see the full Original API Error Object in case the API drifts away from API standardizations.
2.2. Standardized n8n Error Message Display Function This is just a helper function that will be used to ensure the way error messages are displayed to the user is consistent across every single API
const userDisplayNodeError = (nodeName: string, statusCode: string, errorMessage: string): string => {
throw new Error(`${nodeName} error response [${statusCode}]: ${errorMessage}`);
};2.3. Standardized n8n Error Object The Standardized n8n Error Object is an internal n8n representation that can be used to support automatic error workflows.
interface N8nErrorAPI {
code: string[];
message: string[];
}2.4. API Specific Transformers The API Specific Transformer is essentially a way to transform the API's Error Object Structure into n8n's Error Object Structure. This is similar to LoDash's .get function.
const box2ToN8nErrorAPITransformer: N8nErrorAPI = {
code: ["error", "error", "status"],
message: ["error", "error", "message"]
};2.5. Error Handler The Error Handler is just a parser that generates the n8n standard formatted error object.
/**
* Takes in any API's error response object and uses the provided API to n8n Error Transformer to generate an n8n standard formatted error object.
* @param errorObj {any} - The API's Error Response Object
* @param pathTransformer {N8nErrorAPI} - The paths to parse in the errorObj to fetch the correct values.
* @return {N8nErrorAPIResponse} - The n8n standard API error object
*/
const errorHandler = (errorObj: any, pathTransformer: N8nErrorAPI): N8nErrorAPIResponse => { // tslint:disable-line:no-any
let errorObjCopy: any; // tslint:disable-line:no-any
const apiStandard: N8nErrorAPIResponse = {code: '', message: ''};
for (const [key, value] of Object.entries(pathTransformer)) {
errorObjCopy = errorObj;
value.forEach((entry: string) => {
errorObjCopy = errorObjCopy[entry];
});
apiStandard[key] = errorObjCopy.toString();
}
return apiStandard;
};2.6 V1.0 Versus V2.0
| V1.0 | V2.0 |
if (error.statusCode === 401) {
// Return a clear error
throw new Error('The Spotify credentials are not valid!');
}
if (error.statusCode === 403 && error.response.body.message === 'Player command failed: Premium required') {
throw new Error('You must have Spotify Premium for this operation!');
}
if (error.response && error.response.body && error.response.body.message) {
// Try to return the error prettier
throw new Error(`Spotify error response [${error.statusCode}]: ${error.response.body.message}`);
}
// If that data does not exist for some reason return the actual error
throw error; |
const box2ToN8nErrorAPITransformer: N8nErrorAPI = {
code: ["error", "error", "status"],
message: ["error", "error", "message"]
};
const errorObj = errorHandler(error, box2ToN8nErrorAPITransformer);
userDisplayNodeError("Spotify", errorObj.code, errorObj.message); |
4.1 Shared getResourceList Function This function will get a list of all the resources and their respective documentation through the use of JSDocs Parser an n8n node contains.
4.2 Shared getOperationsOfResourceList Function This function will get a list of all the operations and their respective documentation through the use of JSDocs Parser of a specific resource for an n8n node.
4.3 Shared getNodeResourcesAndOperationsList Function This function will loop through the two functions above to get a JSON object that can be parsed for documentation generation.
The way nodes will be deployed will not be modified in the V2.0 architecture.
Since in V2.0, the declaration of resources and operations are more consistent and the evaluation of which resource/operation is being acted upon is centralized by n8n's shared logic. The size should significantly decrease per node addition and the performance should increase accordingly as well. While the size decrease and performance increase are both minimal this won't be the motivating factor of this proposed architecture.
In V2.0, the quality of the code as well as the maintenance of the nodes in n8n will be quite a substantial improvement and thus will be the main motivating factor of this proposed architecture. With fewer bugs, easier updates, and less duplicate code, n8n will be set up to scale to its next tens of thousands of API integrations available!