Mazha.md

Mazha: A Reactive TypeScript Notebook

1. Core Objective & "North Star"

• Objective: To provide a reactive notebook environment specifically designed for TypeScript, enabling users to create and manage notebooks programmatically, with a strong emphasis on type safety and developer experience.
• "North Star" - Deep Project Integration: Mazha's primary differentiator is its deep, native integration with a user's existing TypeScript project environment. This is paramount.
  • tsconfig.json Adherence: Mazha's backend (for execution and module resolution) and frontend (for client-side LSP) will strictly respect the project's tsconfig.json, including paths, baseUrl, compilerOptions, jsxImportSource, etc., ensuring consistent behavior with the rest of the user's project.
  • Seamless Local Imports: Users can import modules, functions, types, and React components directly from their local project files (e.g., ./src/utils, ../components/MyWidget) into Mazha cells, with full type safety and autocompletion provided by the integrated tooling.

2. File Format & Project Setup

• Notebook File: Notebooks are TypeScript files with a .mazha.tsx extension, explicitly supporting JSX syntax within cells. The file itself contains the programmatic definition of the notebook.
• Installation: Mazha is installed as a project dependency: npm install mazha.
• CLI Invocation: Mazha is run via a CLI command, targeting a single notebook file for a given server instance: npx mazha serve ./path/to/my-notebook.mazha.tsx.
  • Initial Focus: Robust single-notebook experience. Multi-notebook management within a single Mazha UI instance is deferred.
• Configuration: An optional mazha.config.ts at the project root can provide project-level settings (e.g., server port, specific backend configurations).

3. System Architecture

• Frontend: React Single Page Application (SPA).
  • Code Editor: CodeMirror 6, configured for TypeScript/TSX.
  • Markdown Editor: Tiptap, providing a Notion-like WYSIWYG experience.
• Backend: Node.js server.
  • Manages the *.mazha.tsx file (reading, writing via AST manipulation).
  • Hosts the Notebook engine for cell processing, execution, and reactivity.
• Frontend-Backend Communication:
  • Initial MazhaContext Load: The comprehensive MazhaContext (type definitions) is fetched via a REST API call upon notebook load.
  • Real-time Updates: Bidirectional communication for all dynamic operations (cell edits, execution requests/results, MazhaContext deltas, status updates) occurs via WebSockets.

4. Notebook Engine (Backend Core)

• Notebook Class: The central class, instantiated by the user in their *.mazha.tsx file (e.g., const notebook = new Notebook<MazhaContext>(); export default notebook;). It manages the collection of cells, their state, the dependency graph, and the execution flow.
• MazhaContext Type: A dynamically generated TypeScript type representing the collective shape of all top-level definitions (variables, functions, classes, types) exported or made available by all cells in the notebook. This type is crucial for inter-cell type safety and autocompletion. It's inferred by the backend using ts-morph analysis of cell outputs.
• Cell Definition (Programmatic):
  • Users define cells by calling methods on the notebook instance within the *.mazha.tsx file.
  • TypeScript/TSX Cells: notebook.cell("uniqueCellId", async ({ dep1, dep2 }) => { /* cell code */ return { outputVar: ... }; }); Dependencies are destructured from the input object. Returned object's properties become outputs.
  • Markdown Cells: notebook.markdownCell("uniqueMarkdownId", "## Markdown Content"); (Mechanism for reactive template literals in Markdown TBD).

5. Reactivity & Dependency Management (Backend)

• Automatic Re-evaluation: When a cell's code changes or its upstream dependencies' outputs change, the cell and its downstream dependents are re-evaluated.
• Dependency Graph: A directed acyclic graph (DAG) of cells.
  • Construction: Built by the backend via static analysis of cell code using ts-morph. It identifies defined variables/outputs per cell and referenced variables (dependencies) from other cells.
• Execution Order: Determined by a topological sort of the dependency graph.
• Circular Dependency Handling:
  • Detection: Implemented using a graph traversal algorithm (e.g., DFS) during graph construction/update.
  • Resolution: Cells involved in a cycle are not executed and are marked with an error state. The UI will indicate the problematic cells and the cycle.
• Limitations: The reliance on static analysis means highly dynamic, runtime-determined dependencies might not be detected. This will be documented.

6. Cell Execution Model (Backend)

• Execution Environment: Cell code (for TypeScript cells) is executed within the Node.js environment using the AsyncFunction constructor, providing some scope control.
• Inputs & Outputs:
  • Inputs: Resolved dependency values are passed as an object to the cell's async function.
  • Outputs: The cell's async function must return an object. The properties of this object become the named outputs of the cell, contributing to the MazhaContext. ts-morph helps identify these intended outputs.
• State Management: The Notebook instance on the backend maintains the canonical state of all cell outputs.

7. Frontend Cell UI & Interaction

• TypeScript Code Cells:
  • CodeMirror 6 editor with TSX support.
  • "Run" button, contextual toolbar (delete, add cell), status indicator.
  • Output area appears dynamically below the editor if the cell produces visual output.
• Markdown Cells:
  • Tiptap editor for WYSIWYG experience (contextual formatting menu, /slash commands).
  • No explicit "Run" button; content renders live. Reactive updates from code cells trigger re-renders.
  • Contextual toolbar.

8. Client-Side Language Server (LSP) for TypeScript Cells

• LSP Environment: The TypeScript language service (typescript package) runs inside a Web Worker in the browser.
• Editor-LSP Integration: CodeMirror 6 (LSP client) communicates with the LSP server in the Web Worker via codemirror-languageserver (or similar).
• MazhaContext for LSP & Virtual File System:
  • @typescript/vfs: A virtual file system is created in the frontend using @typescript/vfs.
  • mazha-globals.d.ts: The initial MazhaContext (type definitions) fetched from the backend is used to create/update a virtual mazha-globals.d.ts file within this VFS.
  • Delta Updates: Subsequent changes to MazhaContext are sent from the backend as deltas (e.g., { action: "add", declaration: "declare const newVar: string;" }) via WebSockets. The frontend applies these deltas to the mazha-globals.d.ts content in the VFS.
  • LSP Notification: The LSP worker is notified of changes to the virtual mazha-globals.d.ts, prompting it to re-evaluate types and provide updated autocompletion and diagnostics.
• Features: Provides rich autocompletion (aware of MazhaContext and project types) and inline diagnostics (type errors, syntax errors) directly in the CodeMirror editor.

9. Output Rendering

• JSX Rendering: An API like mazha.jsx(<MyComponent {...props} />) within a cell allows returning React components. These are serialized/described by the backend and rendered by React on the client-side.
• Markdown Rendering: Handled directly by Tiptap on the client-side.
• Other Outputs: Console logs can be captured and displayed; future support for custom renderers.

10. Error Handling

• Backend Execution Errors: Captured by the Notebook engine. Error details (message, stack, cell ID) are sent to the frontend via WebSockets and displayed in the relevant cell's output area. Dependent cells are typically not run or also enter an error state.
• Frontend Type/Syntax Errors: Detected live by the client-side LSP. Displayed as inline squiggly underlines and hover messages within the CodeMirror editor before execution.

11. UI-Driven Notebook File Updates (The Core Edit Loop)

1. Frontend Action & WebSocket Message: User interaction (e.g., adding a cell, editing code, deleting a cell, reordering cells) triggers a specific WebSocket message to the backend. The message contains the operation type and necessary payload (e.g., { op: "update_cell_code", payload: { cell_id: "cell123", new_code: "..." } }).
2. Backend: Internal Model Update & AST Manipulation:
   • The backend updates its internal, in-memory representation of the notebook.
   • Crucially, it then uses ts-morph to parse the existing *.mazha.tsx file's AST, programmatically make the corresponding changes (e.g., add/remove/modify notebook.cell(...) calls or their arguments), and then save the modified AST back to the *.mazha.tsx file on disk. This ensures the file is always the source of truth.

12. Backend Refresh Cycle (Post File Modification)

1. Re-Analyze Notebook File: After saving the *.mazha.tsx file, the backend immediately re-parses this updated file using ts-morph.
2. Update Internal State: It rebuilds its internal Notebook model, including the cell dependency graph, and re-infers the global MazhaContext.
3. Identify & Execute Cells: It determines the set of cells that need re-evaluation (the directly changed cell, its downstream dependents, any newly added cells). These cells are then executed in the correct topological order.
4. Send Updates to Frontend: The backend sends WebSocket messages to the frontend containing:
   • New outputs for the re-run cells.
   • The updated MazhaContext (ideally as deltas).
   • Any new error states.

13. Frontend Response to Backend Updates (Consuming the Refresh)

1. Process WebSocket Messages: A central WebSocket message handler in the frontend receives and routes incoming messages based on their operation type.
2. Update Client-Side State & UI:
   • Cell Outputs/Status: Updates the state for individual cells (output content, execution status, errors). React re-renders the affected cell components.
   • MazhaContext: Applies received deltas to the mazha-globals.d.ts file within the @typescript/vfs environment and notifies the LSP worker.
   • Notebook Structure: If the overall structure changes (e.g., cell order), the frontend updates its cell list, triggering a broader UI refresh.

14. WebSocket Reliability Strategies

• Client-Side Connection State Management: Maintain and expose states (e.g., CONNECTING, CONNECTED, DISCONNECTED, RECONNECTING).
• Automatic Reconnection: Implement exponential backoff with jitter for reconnection attempts. Limit retries for non-transient server-indicated close reasons.
• Full State Synchronization on (Re)Connection: Upon any successful WebSocket connection, the backend must be capable of sending a "full state" message to ensure the frontend is perfectly synchronized (all cell definitions, current outputs, full MazhaContext, statuses).
• UI Feedback: Provide clear, persistent visual indicators of connection status. Use toasts/notifications for transient events and modals for critical disconnections.
• Heartbeating (Keep-Alive): Consider client-server pings/pongs to detect silent connection drops.

15. Key Deferred/Future Considerations

• Advanced expensive cell management (caching strategies, manual execution flags).
• Detailed serialization and communication for mazha.jsx() outputs.
• Offline support and edit queuing during disconnections.
• Multi-notebook management UI.
• Fine-grained permissions or sandboxing if notebooks are ever run in less trusted environments.
• Extensible renderer plugin system.