DRAFT jupyter(lab)-lsp incorporation pre-proposal

preproposal.md

Pre-Proposal Checklist

• Briefly outlines the proposal
• Suggests a review team (optional)
• Declares why it should be a JEP (See the “JEP / Not-a-JEP Rubric” below.)
• Identify a Shepherd to see the process through. (Shepherds are assigned on a round-robin basis from a set of interested engaged volunteers).
• Decide if it’s a JEP according to the criteria listed above. The Shepherd decides if the JEP criteria have been met.

Pre-Proposal

The maintainers of the jupyter(lab)-lsp would like to propose its incorporation as an official Project Jupyter project, as signalled by a new, Jupyter-branded GitHub organization. We feel it would benefit Jupyter users through higher velocity in delivering more robust features otherwise missing in the Jupyter interactive computing experience.

The key component of the repository, @krassowski/jupyterlab-lsp, offers Jupyter users an expanding subset of features described by the Language Server Protocol (LSP) as an extension to JupyterLab. These features include refinements of existing Jupyter interactive computing features, such as completion and introspection, as well as new Jupyter features such as linting, reference linking, and symbol renaming. It is supported by jupyter-lsp, a Language Server- and Jupyter Client-agnostic extension of the Jupyter Notebook Server. We will discuss the architecture and engineering process of maintaining these components at greater length, leveraging a good deal of the user and developer documentation.

Much like Jupyter Kernel Messaging, LSP provides a language-agnostic, JSON-compatible description for multiple clients to integrate with any number of language implementations. Unlike Kernel Messaging, the focus is on precise definition of the many facets of static analysis and code transformation, with nearly four times the number of messages of the Jupyter specification. We will discuss the opportunities and challenges of this complexity on users and maintainers of Jupyter Clients and Kernels.

Kernel integration

In order for jupyterlab-lsp to be able to connect to the appropriate language server, it requires that kernels properly implemenet KernelInfo from version 5.0 of Jupyter messaging specification, in particular the field language_info with the following fields present:

• mimetype
• file_extension
• name

Therefore, no changes to the specification are required. Any specification change ideas will be sent as separate JEPs.

We support kernels for all three of the Jupyter-name forming languages:

• Julia: IJulia using LanguageServer.jl,
• Python: IPython using pyls or jedi-language-server,
• R: IRKernel using R languageserver

Other kernels were also confirmed to work straightaway (Bash) or after minimal user configuration (Scala) in presence of appropriate language servers. Academic prototype works were created using the LSP as a connector.

Magics and polyglot support

We also support the magics in IPython, allowing for polyglot experience when using cell magics. For the detailed discussions on magics adoptions please refer to https://github.com/krassowski/jupyterlab-lsp/issues/347.

Outlook towards notebook-specific IDE features

The team envisions a support for the notebook specific IDE features to be included in the future. This might involve upstream changes to the LSP specification, or custom jupyter-specific extension. Please see our discussion in https://github.com/krassowski/jupyterlab-lsp/issues/467 for more details.

Minimal implementation details

• jupyterlab-lsp implements integration with JupyterLab and its default CodeMirror editor; the design allows for incorporation of additional editors in the future, but large portions of code are CodeMirror-specific; it communicates with jupyter-lsp to send/recieve messages from language servers and with JupyterLab to get information about currrent kernel
• jupyter-lsp uses jupyter_server, tornado, sockets and IO streams for communication with servers and with frontend; it is not aware of kernels existence
• we use GitHub CI running pytest, jest and robot-framework (selenium) test; the integration/acceptance tests for the bulk of the frontend testing; we test on Windows, Linux and MacOS, supporting thee Python versions at a time for jupyter-lsp and doing tests with actual language servers

Previous discussions:

Please refer to:

• jupyterlab/frontends-team-compass#67 for the discussion held within the JupyterLab team.
• jupyter/enhancement-proposals#26 for previous (stale) JEP on LSP adoption; this proposal superseds that JEP; importantly jupyter(lab)-lsp does not require any specific editor, whereas the previous JEP specifically mentions Monaco.