Jupyter[Lab] Language Server Protocol

This is the documentation for:

• jupyterlab-lsp

• jupyter-lsp

Both are managed on GitHub, where you can find the issue tracker.

Installation

Please Read This First

Delivering LSP features to your JupyterLab requires three pieces:

1. jupyter-lsp

• runs in your jupyter_server application on your server to handle requests from the browser to language servers

• to run, you need:

  • python >=3.8,<3.11.0a0

  • jupyter_server >=1.1.2

2. jupyterlab-lsp

• runs in your browser, as an extension to JupyterLab

• to install it, you need:

  • jupyterlab >=4.0.6,<5.0.0a0

3. Language Servers

• run on your server

  • probably in another language runtime than python

• some can be automatically detected if installed

• others also need to be configured

Clean installation

The approaches demonstrated below will ensure that correct versions of Python, JupyterLab and extensions are installed, and are generally recommended for novice users, or users who are not familiar with the Python ecosystem. However, if you feel proficient with using Python package managers such as pip and conda you may prefer to follow the custom installation guide instead, which allows to install the packages in an existing environment.

conda (minimal python)

When installing from conda-forge, the jupyter-lsp-python bundle includes both the server extension (jupyter-lsp) and pyls (a third-party server also known as python-language-server). You can swap jupyter-lsp-python with another pre-made bundle, jupyter-lsp-r to get get the same server extension but with r-languageserver instead. Alternatively, you can install a language server of your choice manually (see further steps).

%%markdown
```bash
conda create -c conda-forge -n lsp 'python {REQUIRED_PYTHON}' 'jupyterlab={JUPYTERLAB_VERSION}' 'jupyterlab-lsp={JUPYTERLAB_LSP_VERSION}' 'jupyter-lsp-python={JUPYTER_LSP_VERSION}'
conda activate lsp
```

Then run

```bash
jupyter lab
```

Your browser should open to your local server.

conda create -c conda-forge -n lsp 'python >=3.8,<3.11.0a0' 'jupyterlab=4.0.6' 'jupyterlab-lsp=5.0.3' 'jupyter-lsp-python=2.2.2'
conda activate lsp

Then run

jupyter lab

Your browser should open to your local server.

Custom installation

Get a working JupyterLab environment

Refer to the official JupyterLab Installation Documentation for your installation approach.

Verify your lab works:

jupyter lab --version
jupyter lab

Install Jupyter[Lab] LSP

conda

conda install jupyterlab-lsp=5.0.3

pip

pip install jupyterlab-lsp==5.0.3

Next step: Language Servers

Now that you have jupyterlab-lsp, jupyter-lsp and all of their dependencies, you’ll need some language servers. See:

• Language Servers that will be found automatically once installed

• configuring jupyter-lsp for more control over which servers to load

Language servers

By default jupyter-lsp does not come with any language servers preinstalled. However, we will try to use them if they are installed and we know about them (i.e. someone contributed a full specification).

You can disable auto-detection by configuring autodetect

You can add another language server for languages that are not listed on this page:

• using a minimal JSON or Python configuration file (good for experimenting or configuring a niche server), or

• contributing a full specification (to enable better integration and help other users of the same language)

The existing language servers are listed on the official list and on the community-curated list.

For the language servers in the tables below, use one of the suggested package managers to install them: these implementations are tested to work with jupyter-lsp.

Servers tested with Notebooks

While most servers should work with notebooks and standalone files, some require additional adjustments. The following language servers are tested against jupyterlab-lsp and any issues when using them in notebooks will be prioritised.

If you choose to install multiple language servers for the same language, the one with the highest priority (which can be set in the Advanced Settings Editor) will be used.

Languages	Implementation	Installation
python	jedi-language-server	pip: pip install -U jedi-language-server conda: conda install -c conda-forge jedi-language-server
julia	julia-language-server	julia: using Pkg; Pkg.add("LanguageServer")
python	pylsp	pip: pip install 'python-lsp-server[all]' conda: conda install -c conda-forge python-lsp-server
r	r-languageserver	cran: install.packages("languageserver") conda: conda install -c conda-forge r-languageserver

If you plan to add a custom language server for the use with notebooks, please note that a complete set of information should be provided by the kernel, as described in making custom servers work with notebooks.

Servers tested with file editor

Servers requiring Node.js

These servers have mostly been tested with file editors.

Languages	Implementation	Installation
bash sh	bash-language-server	npm: npm install --save-dev bash-language-server yarn: yarn add --dev bash-language-server jlpm: jlpm add --dev bash-language-server
dockerfile	dockerfile-language-server-nodejs	npm: npm install --save-dev dockerfile-language-server-nodejs yarn: yarn add --dev dockerfile-language-server-nodejs jlpm: jlpm add --dev dockerfile-language-server-nodejs
python	pyright	npm: npm install --save-dev pyright yarn: yarn add --dev pyright jlpm: jlpm add --dev pyright
sql	sql-language-server	npm: npm install --save-dev sql-language-server yarn: yarn add --dev sql-language-server jlpm: jlpm add --dev sql-language-server
javascript jsx typescript typescript-jsx typescriptreact javascriptreact	typescript-language-server	npm: npm install --save-dev typescript-language-server yarn: yarn add --dev typescript-language-server jlpm: jlpm add --dev typescript-language-server
markdown ipythongfm gfm	unified-language-server	npm: npm install --save-dev unified-language-server yarn: yarn add --dev unified-language-server jlpm: jlpm add --dev unified-language-server
css less scss	vscode-css-languageserver-bin	npm: npm install --save-dev vscode-css-languageserver-bin yarn: yarn add --dev vscode-css-languageserver-bin jlpm: jlpm add --dev vscode-css-languageserver-bin
html	vscode-html-languageserver-bin	npm: npm install --save-dev vscode-html-languageserver-bin yarn: yarn add --dev vscode-html-languageserver-bin jlpm: jlpm add --dev vscode-html-languageserver-bin
json	vscode-json-languageserver-bin	npm: npm install --save-dev vscode-json-languageserver-bin yarn: yarn add --dev vscode-json-languageserver-bin jlpm: jlpm add --dev vscode-json-languageserver-bin
yaml	yaml-language-server	npm: npm install --save-dev yaml-language-server yarn: yarn add --dev yaml-language-server jlpm: jlpm add --dev yaml-language-server

NodeJS (preferably even-numbered an Active or Maintenance Long Term Support release) is a prerequisite for installation of any of the above language servers; you can get it with:

conda install --channel conda-forge nodejs
# or one of the following, as an administrator
choco install nodejs            # Windows with Chocolatey
sudo apt-get install nodejs     # Debian/Ubuntu
sudo brew install nodejs        # MacOS with Homebrew
sudo dnf install nodejs         # Fedora
sudo yum install nodejs         # RHEL/CentOS

Example: getting all the Node.js-based language servers

A number of language servers are built on the reference implementation, powered by Node.js. The most reliable place to install these is in a node_modules in the directory where you launch jupyter lab.

For example, to install all the servers which are tested as part of jupyterlab-lsp:

jlpm add --dev  \
    bash-language-server \
    dockerfile-language-server-nodejs \
    pyright \
    sql-language-server \
    typescript-language-server \
    unified-language-server \
    vscode-css-languageserver-bin \
    vscode-html-languageserver-bin \
    vscode-json-languageserver-bin \
    yaml-language-server

This will create (or add to):

• package.json (check this in!)

• yarn.lock (check this in!)

• node_modules/ (add to your VCS ignore file)

If you wish to install these someplace else, you may need to specify where you install them with extra_node_roots.

Standalone servers

These servers have been mostly tested with file editor.

Languages	Implementation	Installation
tex latex	texlab	conda: conda install -c conda-forge texlab chktex

Example: Getting a \(\LaTeX\) stack

conda install --channel conda-forge tectonic texlab chktex

This will install:

• tectonic, a cross-platform \(\LaTeX\) processing tool

  • note, it will download a large number of packages when first executed

• texlab, a Language Server for .tex files that supports completion and reference navigation

• chktex, a .tex style checker

Community-supported servers

Servers and extensions listed below are not included in the testing suite of jupyterlab-lsp; the support may be provided by community members or a third-party:

Languages	Implementation	Comment
Spark SQL	CybercentreCanada/jupyterlab-sql-editor
GraphQL, SPARQL, Turtle	jupyrdf/graph-lsp
JSON	jupyter-lsp/json-lsp	(1)
YAML	jupyter-lsp/yaml-lsp	(1)
Scala	scalameta/metals	(2)

• (1) convenience wrappers for installation with pip, require Node.js.

• (2) metals is not currently auto-detected, but can be configured as demonstrated in the configuration example.

Troubleshooting

r-languageserver

Rscript not found. Please ensure that RScript executable is in the PATH; this should happen automatically when using Linux, Mac OS or Conda, but will require manual configuration when using the default R installer on Windows. For more details please consult documentation: https://cran.r-project.org/bin/windows/base/rw-FAQ.html#Rcmd-is-not-found-in-my-PATH_0021 If Rscript is already in the PATH, you can check whether the language server package is properly installed with: Rscript -e "cat(system.file(package='languageserver'))" which should return the path to the installed package.

texlab

texlab not found. Please ensure that texlab executable is in the PATH; this should happen automatically when installing texlab from Conda, but may require manual configuration of PATH environment variable if you compiled texlab from source. You can ensure check if texlab is in the PATH, by running: which texlab which should return the path to the executable (if found).

Configuring backend

Configuration Files

Like the Jupyter Notebook server, JupyterHub, and other Jupyter interactive computing tools, jupyter-lsp can be configured via Python or JSON files in well-known locations. You can find out where to put them on your system with:

jupyter --paths

They will be merged from bottom to top, and the directory where you launch your notebook or lab server wins, making it easy to check in to version control.

Configuration Options

language_servers

jupyter-lsp does not come with any Language Servers! However, we will try to use known language servers if they are installed and we know about them. You can disable auto-detection behavior by configuring autodetect.

If you did not find an implementation for the language server you need on the list of known language servers, continue reading!

Please consider contributing your language server spec to jupyter-lsp!

The absolute minimum language server spec requires:

• argv, a list of shell tokens to launch the server in stdio mode (as opposed to tcp),

  • shell tokens are arrays of strings representing command line commands with arguments, for example ls -l is represented as ["ls", "-l"] while mkdir "new directory" should be split into ["mkdir", "new directory"]; If you have Python installed, you can use shlex.split("your command") to get such an array.

• the languages which the language server will respond to, and

• the schema version of the spec (currently 2)

• mime_types by which the notebooks and files will be matched to the language server:

  • for notebooks the MIME type is derived from language_info/mimetype element of kernel_info response (with fallback on to cell metadata if missing from kernel response)

  • for files the implementation is frontend-specific; in JupyterLab the MIME type is obtained from: a) the code editor MIME type registry, which is by default using the CodeMirror mode as for JupyterLab 3.x, or if no specific MIME type is found there, then b) from the DocumentRegistry file type by matching the contentsModel against the registered file types using getFileTypeForModel() method (if multiple MIME types are present, the first one will be used).

• requires_documents_on_disk should be false for all new specifications, as any code paths requiring documents on disks should be fixed in the LSP servers rather than masked by using the .virtual_documents workaround.

# ./jupyter_server_config.json                   ---------- unique! -----------
#                                               |                              |
# or e.g.                                       V                              V
# $PREFIX/etc/jupyter/jupyter_server_config.d/a-language-server-implementation.json
{
  "LanguageServerManager": {
    "language_servers": {
      "a-language-server-implementation": {
        "version": 2,
        "argv": ["/absolute/path/to/a-language-server", "--stdio"],
        "languages": ["a-language"],
        "mime_types": ["text/language", "text/x-language"],
        "display_name": "My LSP server"
      }
    }
  }
}

The documentation of display_name along many other properties is available in the schema. Please note that some of the properties defined in the schema are intended for future use: we would like to use them to enrich the user experience but we prioritized other features for now. We welcome any help in creating the user interface making use of these properties.

More complex configurations that can’t be hard-coded may benefit from the python approach:

# jupyter_server_config.py
import shutil

# c is a magic, lazy variable
c.LanguageServerManager.language_servers = {
    "a-language-server-implementation": {
        # if installed as a binary
        "argv": [shutil.which("a-language-server")],
        "languages": ["a-language"],
        "version": 2,
        "mime_types": ["text/a-language"],
        "display_name": "A language server"
    },
    "another-language-implementation": {
        # if run like a script
        "argv": [shutil.which("another-language-interpreter"), "another-language-server"],
        "languages": ["another-language"],
        "version": 2,
        "mime_types": ["text/another-language"],
        "display_name": "Another language server"
    }
}

nodejs

default: None

An absolute path to your nodejs executable. If None, nodejs will be detected in a number of well-known places.

autodetect

default: True

If True, jupyter-lsp will look for all known language servers. User-configured language_servers of the same implementation will be preferred over autodetected ones.

node_roots

default: []

Absolute paths to search for directories named node_modules, such as nodejs-backed language servers. The order is, roughly:

• the folder where notebook or lab was launched

• the JupyterLab staging folder

• wherever conda puts global node modules

• wherever some other conventions put it

extra_node_roots

default: []

Additional places jupyter-lsp will look for node_modules. These will be checked before node_roots, and should not contain the trailing node_modules.

virtual_documents_dir

default: os.getenv(“JP_LSP_VIRTUAL_DIR”, “.virtual_documents”)

Path (relative to the content manager root directory) where a transient copy of the virtual documents should be written allowing LSP servers to access the file using operating system’s file system APIs if they need so (which is discouraged).

Its default value can be set with JP_LSP_VIRTUAL_DIR environment variable. The fallback value is .virtual_documents.

Python entry_points

pip-installable packages in the same environment as the Jupyter server can be automatically detected as providing language_servers. These are a little more involved, but also more powerful: see more in Contributing. Servers configured this way are loaded before those defined in configuration files, so that a user can fine-tune their available servers.

Making Custom Servers Work With Notebooks

To enable integration of language servers with Jupyter notebooks this extensions assumes that the language_info section of kernel_info_reply is complete and properly returned by the Kernel. In particular the following elements are required:

• File extension: many language servers only handle files with specific file extensions and refuse to operate if not provided with such; the file extension of a native script for a given language (this is other than .ipynb), derived from file_extension field of language_info, will be added to the name of the notebook when communicating with the language server to satisfy the file extension check.

• MIME type: matching of notebooks to servers is based on the MIME types declared in server specification files and mimetype field of language_info. If kernel fails to provide any MIME type, connecting the language server will not be possible; if multiple MIME types are in use, any would work well for this extension as long as you also include it in the mime_types list of language server specification.

Example: Scala Language Server (metals) integration

Step 1: Get a Scala-based kernel installed.

2 possible options: Almond kernel or the Spark magic kernel.

Almond kernel install:

$ curl -Lo coursier https://git.io/coursier-cli
$ chmod +x coursier
$ ./coursier launch --fork almond -- --install
$ rm -f coursier

Spark Magic kernel:

pip install sparkmagic

Now, install the spark kernel:

jupyter-kernelspec install sparkmagic/kernels/sparkkernel

Step 2: Install metals server in the working directory:

Metals has a coursier based installation.

curl -Lo coursier https://git.io/coursier-cli && chmod +x coursier
./coursier bootstrap org.scalameta:metals_2.12:0.7.0 --force-fetch -o metals -f

(Might need to use the --force-fetch flag if you are getting dependency issues.)

Step 3: Configure the metals server in jupyterlab-lsp. Enter the following in the jupyter/jupyter_server_config.d/metals-ls.json (in one of the jupyter configuration directories):

{
  "LanguageServerManager": {
    "language_servers": {
      "metals": {
        "version": 2,
        "argv": ["<$path_to_metals_server(eg:/Users/skakker/projects/jupyterlab-lsp/metals)>"],
        "languages": ["scala"],
        "mime_types": ["text/x-scala"]
      }
    }
  }
}

You are good to go now! Just start jupyter lab and create a notebook with either the Spark or the Scala kernel and the metals server should automatically initialise (the status indicator should show “Fully initialized”).

Contributing

jupyter-lsp and jupyterlab-lsp are open source software, and all contributions conforming to good sense, good taste, and the Jupyter Code of Conduct are welcome, and will be reviewed by the contributors, time-permitting.

You can contribute to the project through:

• creating language server specs

  • you can publish them yourself (it might be a single file)…

  • or advocate for adding your spec to the github repository and its various distributions

    • these are great first issues, as you might not need to know any python or javascript

• proposing parts of the architecture that can be extended

• improving documentation

• tackling Big Issues from the future roadmap

• improving testing

• reviewing pull requests

Thank you for all your contributions :heart:

Set up the environment

Development requires, at a minimum:

• nodejs >=16,!=17,<19

• python >=3.8,<3.11.0a0

• jupyterlab >=4.0.6,<5.0.0a0

It is recommended to use a virtual environment (e.g. virtualenv or conda env) for development.

To use the same environment as the binder demo (recommended):

conda env update -n jupyterlab-lsp --file binder/environment.yml # create a conda env
conda activate jupyterlab-lsp                                    # activate it

Or with pip:

pip install -r requirements/dev.txt  # in a virtualenv, probably
sudo apt-get install nodejs          # ... e.g. on debian/ubuntu

Single-step installation

Once your environment is created and activated, on Linux/OSX you can run:

bash binder/postBuild

This performs all the basic setup steps, and is used for the binder demo.

This approach may not always work. Continue reading for a step-by-step instructions which also show all the underlying pieces.

Manual installation

Install jupyter-lsp from source in your virtual environment:

python -m pip install -e python_packages/jupyter_lsp --ignore-installed --no-deps -vv

Enable the server extension:

jupyter server extension enable --sys-prefix --py jupyter_lsp

Install npm dependencies, build TypeScript packages, and link to JupyterLab for development:

jlpm bootstrap
# if you installed `jupyterlab_lsp` before uninstall it before running the next line
jupyter labextension develop python_packages/jupyterlab_lsp/ --overwrite
# optional, only needed for running a few tests for behaviour with missing language servers
jupyter labextension develop python_packages/klingon_ls_specification/ --overwrite

Note: on Windows you may need to enable Developer Mode first, as discussed in jupyterlab#9564

Frontend Development

To rebuild the schemas, packages, and the JupyterLab app:

jlpm build
jupyter lab build

To watch the files and build continuously:

jlpm watch   # leave this running...
jupyter lab --watch  # ...in another terminal

Now after each change to TypesScript files wait until both watchers finish compilation, and then refresh the JupyterLab in your browser.

Note: the backend schema is not included in watch, and is only refreshed by build

To check and fix code style:

jlpm lint

To run test the suite (after running jlpm build or watch):

jlpm test

To run tests matching specific phrase, forward -t argument over yarn and lerna to the test runners with two --:

jlpm test -- -- -t match_phrase

To verify the webpack build wouldn’t include problematic vendored dependencies:

python scripts/distcheck.py

Server Development

Testing jupyter-lsp

python scripts/utest.py

Documentation

To build the documentation:

python scripts/docs.py

To watch documentation sources and build continuously:

python scripts/docs.py --watch

To check internal links in the docs after building:

python scripts/docs.py --check --local-only

To check internal and external links in the docs after building:

python scripts/docs.py --check

Note: you may get spurious failures due to rate limiting, especially in CI, but it’s good to test locally

Browser-based Acceptance Tests

The browser tests will launch JupyterLab on a random port and exercise the Language Server features with Robot Framework and SeleniumLibrary. It is recommended to peruse the Robot Framework User’s Guide (and the existing .robot files in atest) before working on tests in anger.

First, ensure you’ve prepared JupyterLab for jupyterlab-lsp frontend and server development.

Prepare the environment:

conda env update -n jupyterlab-lsp --file requirements/atest.yml

or with pip

pip install -r requirements/atest.txt  # ... and install geckodriver, somehow
sudo apt-get install firefox-geckodriver    # ... e.g. on debian/ubuntu

Run the tests:

python scripts/atest.py

The Robot Framework reports and screenshots will be in atest/output, with <operating system>_<python version>_<attempt>.<log|report>.html and subsequent screenshots being the most interesting artifact, e.g.

atest/
  output/
    linux_37_1.log.html
    linux_37_1.report.html
    linux_37_1/
      screenshots/

Customizing the Acceptance Test Run

By default, all of the tests will be run, once.

The underlying robot command supports a vast number of options and many support wildcards (* and ?) and boolean operators (NOT, OR). For more, start with simple patterns.

Run a suite

python scripts/atest.py --suite "05_Features.Completion"

Run a single test

python scripts/atest.py --test "Works When Kernel Is Idle"

Run test with a tag

Tags are preferable to file names and test name matching in many settings, as they are aggregated nicely between runs.

python scripts/atest.py --include feature:completion

… or only Python completion

python scripts/atest.py --include feature:completionANDlanguage:python

Just Keep Testing with ATEST_RETRIES

Run tests, and rerun only failed tests up to two times:

ATEST_RETRIES=2 python scripts/atest.py --include feature:completion

After running a bunch of tests, it may be helpful to combine them back together into a single log.html and report.html with rebot. Like atest.py, combine.py also passes through extra arguments

python scripts/combine.py

Troubleshooting

• If you see the following error message:

  Parent suite setup failed:
  TypeError: expected str, bytes or os.PathLike object, not NoneType
  

  it may indicate that you have no firefox, or geckodriver installed (or discoverable in the search path).

• If a test suite for a specific language fails it may indicate that you have no appropriate server language installed (see LANGUAGESERVERS)

• If you are seeing errors like Element is blocked by .jp-Dialog, caused by the JupyterLab Build suggested dialog, (likely if you have been using jlpm watch) ensure you have a “clean” lab (with production assets) with:

  jupyter lab clean
  jlpm build
  jlpm lab:link
  jupyter lab build --dev-build=False --minimize=True
  

  and re-run the tests.

• To display logs on the screenshots, configure the built-in ILSPLogConsole console, to use the 'floating' implementation.

• If you see:

  SessionNotCreatedException: Message: Unable to find a matching set of capabilities

  geckodriver >=0.27.0 requires an actual Firefox executable. Several places will be checked (including where conda-forge installs, as in CI): to test a Firefox not on your PATH, set the following environment variable:

  export FIREFOX_BINARY=/path/to/firefox      # ... unix
  set FIREFOX_BINARY=C:\path\to\firefox.exe   # ... windows
  

• If you see Element ... could not be scrolled into view in the Open Context Menu for File step check if you have an alternative file browser installed (such as jupyterlab-unfold) which might interfere with testing (it is recommended to run the tests in an separated environment)

Formatting

Minimal code style is enforced with pytest-flake8 during unit testing. If installed, pytest-black and pytest-isort can help find potential problems, and lead to cleaner commits, but are not enforced during CI tests (but are checked during lint).

You can clean up your code, and check for using the project’s style guide with:

python scripts/lint.py

Specs

While language servers can be configured by the user using a simple JSON or Python configuration file, it is preferable to provide users with an option that does not require manual configuration. The language server specifications (specs) wrap the configuration (as would be defined by the user) into a Python class or function that can be either:

• distributed using PyPI/conda-forge and made conveniently available to users for pip install and/or conda install

• contributed to the collection of built-in specs of jupyter-lsp by opening a PR (preferable for popular language servers, say >100 users)

In either case the detection of available specifications uses Python entry_points (see the [options.entry_points] section in jupyter-lsp setup.cfg).

If an advanced user installs, locates, and configures, their own language server it will always win vs an auto-configured one.

Writing a spec

A spec is a Python callable (a function, or a class with __call__ method) that accepts a single argument, the LanguageServerManager instance, and returns a dictionary of the form:

{
  "python-language-server": {            # the name of the implementation
      "version":  SPEC_VERSION,          # the version of the spec schema (an integer)
      "argv": ["python", "-m", "pyls"],  # a list of command line arguments
      "languages": ["python"],           # a list of languages it supports
      "mime_types": ["text/python", "text/x-ipython"]
  }
}

The above example is only intended as an illustration and not as an up-to-date guide. For details on the dictionary contents, see the schema definition and built-in specs. Basic concepts (meaning of the argv and languages arguments) are also explained in the configuration files documentation.

When contributing a specification we recommend to make use of the helper classes and other utilities that take care of the common use-cases:

• ShellSpec helps to create specs for servers that can be started from command-line

• PythonModuleSpec is useful for servers which are Python modules

• NodeModuleSpec will take care of finding Node.js modules

See the built-in built-in specs for example implementations.

The spec should only be advertised if the command could actually be run:

• its runtime/interpreter (e.g. julia, nodejs, python, r, ruby) is installed

• the language server itself is installed (e.g. python-language-server)

otherwise an empty dictionary ({}) should be returned.

Common Concerns

• some language servers need to have their connection mode specified

  • the stdio interface is the only one supported by jupyter_lsp

    • PRs welcome to support other modes!

• many language servers use nodejs

  • LanguageServerManager.nodejs will provide the location of our best guess at where a user’s nodejs might be found

• some language servers are hard to start purely from the command line

  • use a helper script to encapsulate some complexity, or

  • use a command argument of the interpreter is available (see the r spec and julia spec for examples)

Example: making a pip-installable cool-language-server spec

Consider the following (absolutely minimal) directory structure:

- setup.py
- jupyter_lsp_my_cool_language_server.py

You should consider adding a LICENSE, some documentation, etc.

Define your spec:

# jupyter_lsp_my_cool_language_server.py
from shutil import which


def cool(app):
    cool_language_server = shutil.which("cool-language-server")

    if not cool_language_server:
        return {}

    return {
        "cool-language-server": {
            "version": 1,
            "argv": [cool_language_server],
            "languages": ["cool"],
            "mime_types": ["text/cool", "text/x-cool"]
        }
    }

Tell pip how to package your spec:

# setup.py
import setuptools
setuptools.setup(
    name="jupyter-lsp-my-cool-language-server",
    py_modules=["jupyter_lsp_my_cool_language_server"],
    entry_points={
        "jupyter_lsp_spec_v1": [
            "cool-language-server = jupyter_lsp_my_cool_language_server:cool"
        ]
    }
)

Test it!

python -m pip install -e .

Build it!

python setup.py sdist bdist_wheel

Debugging

To see more see more log messages navigate to Settings ❯ Settings Editor ❯ Language Servers and adjust:

• adjust Logging console verbosity level

• switch Ask servers to send trace notifications to verbose

• toggle Log all LSP communication with the LSP servers

For robot tests set:

Configure JupyterLab Plugin  {"loggingConsole": "floating", "loggingLevel": "debug"}

Extend jupyterlab-lsp

How to add a new LSP feature?

Features (as well as other parts of the frontend) reuse the JupyterLab plugins system. Each plugin is a TypeScript package exporting one or more JupyterFrontEndPlugins (see the JupyterLab extesion developer tutorial for an overview).

Each feature has to register itself with the FeatureManager (which is provided after requesting ILSPFeatureManager token from @jupyterlab/lsp) using register(options: IFeatureOptions) method.

The feature specification should follow the IFeature interface as of JupyterLab 4.0, including:

• id: unique identifier of the feature, we recommend @organization/project:feature pattern

• capabilities: an optional object defining the client capabilities implemented by your feature,

See JupyterLab Extension Points >> LSP Features documentation for more details.

How to override the default implementation of a feature?

You can specify a list of plugins implementing features which you want to disable in jupyterlab.disabledExtensions stanza of package.json, for example:

"jupyterlab": {
  "disabledExtensions": ["@jupyter-lsp/jupyterlab-lsp:hover"]
}

will disable the hover feature.

How to add a custom magic or foreign extractor?

It is now possible to register custom code replacements using ILSPCodeOverridesManager token and to register custom foreign code extractors using ILSPCodeExtractorsManager token, however this API is considered provisional and subject to change.

Future plans for transclusions handling

We welcome pull requests enabling kernels to register their custom syntax/code transformations. The frontend API will remain available for the end-users who write their custom syntax modifications with actionable side-effects (e.g. a custom IPython magic which copies a variable from the host document to the embedded document).

How to add custom icons for the completer?

1. Prepare the icons in the SVG format (we use 16 x 16 pixels, but you should be fine with up to 24 x 24). You can load them for webpack in typescript using imports if you include a typings.d.ts file with the following content:

   declare module '*.svg' {
     const script: string;
     export default script;
   }
   

   in your src/. You should probably keep the icons in your style/ directory.

2. Prepare CompletionKind → IconSvgString mapping for the light (and optionally dark) theme, implementing the ICompletionIconSet interface. We have an additional Kernel completion kind that is used for completions provided by kernel that had no recognizable type provided.

3. Provide all other metadata required by the ICompletionTheme interface and register it on ILSPCompletionThemeManager instance using register_theme() method.

4. Provide any additional CSS styling targeting the JupyterLab completer elements inside of .lsp-completer-theme-{id}, e.g. .lsp-completer-theme-material .jp-Completer-icon svg for the material theme. Remember to include the styles by importing the in one of the source files.

For an example of a complete theme see theme-vscode.

Migrating to v5.0

• IFeature interface was moved to @jupyterlab/lsp

  • labIntegration was removed,

  • editorIntegrationFactory was removed in JupyterLab 4.0 and restored in JupyterLab 4.1 as extensionFactory with new API (ILSPEditorExtensionFactory),

  • supersedes was removed; you can disable extensions using the JupyterLab native jupyterlab.disabledExtensions stanza of package.json.

• ILSPCompletionThemeManager:

  • register_theme() was renamed to registerTheme()

  • all other methods were renamed to follow camelCase convention

Extend jupyter-lsp

Language Server Specs

Language Server Specs can be configured by Jupyter users, or distributed by third parties as python or JSON files. Since we’d like to see as many Language Servers work out of the box as possible, consider contributing a spec, if it works well for you!

Message Listeners

Message listeners may choose to receive LSP messages immediately after being received from the client (e.g. jupyterlab-lsp) or a language server. All listeners of a message are scheduled concurrently, and the message is passed along once all listeners return (or fail). This allows listeners to, for example, modify files on disk before the language server reads them.

If a listener is going to perform an expensive activity that shouldn’t block delivery of a message, a non-blocking technique like IOLoop.add_callback and/or a queue should be used.

Add a Listener with entry_points

Listeners can be added via entry_points by a package installed in the same environment as notebook:

## setup.cfg

[options.entry_points]
jupyter_lsp_listener_all_v1 =
  some-unique-name = some.module:some_function
jupyter_lsp_listener_client_v1 =
  some-other-unique-name = some.module:some_other_function
jupyter_lsp_listener_server_v1 =
  yet-another-unique-name = some.module:yet_another_function

At present, the entry point names generally have no impact on functionality aside from logging in the event of an error on import.

Add a Listener with Jupyter Configuration

Listeners can be added via traitlets configuration, e.g.

## jupyter_server_config.jsons
{
  'LanguageServerManager':
    {
      'all_listeners': ['some.module.some_function'],
      'client_listeners': ['some.module.some_other_function'],
      'server_listeners': ['some.module.yet_another_function']
    }
}

Add a listener with the Python API

lsp_message_listener can be used as a decorator, accessed as part of a serverextension.

This listener receives all messages from the client and server, and prints them out.

from jupyter_lsp import lsp_message_listener

def load_jupyter_server_extension(nbapp):

    @lsp_message_listener("all")
    async def my_listener(scope, message, language_server, manager):
        print("received a {} {} message from {}".format(
          scope, message["method"], language_server
        ))

scope is one of client, server or all, and is required.

Listener options

Fine-grained controls are available as part of the Python API. Pass these as named arguments to lsp_message_listener.

• language_server: a regular expression of language servers

• method: a regular expression of LSP JSON-RPC method names

Releasing

jupyterlab-lsp and jupyter-lsp releases may require building both the python package and nodejs packages.

Updating Version Strings

Use the bump_versions script to manage the version strings:

python scripts/bump_versions.py

Check the version strings across the various files:

python scripts/integrity.py

This repository does not yet support jupyter-releaser but pull requests are welcome!

Publishing

Note: “smoke source install” step on CI will fail when bumping internal dependencies until those are published on NPM. Make sure to release those first and re-run this check then.

cd packages/completion-theme
npm publish --access public
cd -
cd packages/theme-material
npm publish --access public
cd -
cd packages/theme-vscode
npm publish --access public
cd -
cd packages/jupyterlab-lsp
npm publish --access public
cd -
./scripts/publish_pypi.sh

Changelog

@jupyter-lsp/jupyterlab-lsp 5.0.3

• bug fixes:

  • fix nested transclusions in JupyterLab 4.0.7+ (#1045)

  • fix completions when type is not defined (#1044)

@jupyter-lsp/jupyterlab-lsp 5.0.2

• bug fixes:

  • fix native JupyterLab completion not working when LSP completion plugin is disabled (#1036)

jupyter-lsp 2.2.2

• bug fixes:

  • address warning about renamed extension_points (#1035)

  • fix compatibility with jupyter server 1.x

  • fix an authentication-related security vulnerability (see the advisory for details)

• enhancements:

  • add authorization support (lsp resource, jupyter-server v2+ only) - this allows server operators for fine grained access control, e.g. in case if specific users (such as guest or read-only users) should not be allowed to access LSP; this is in addition to authentication fixes

@jupyter-lsp/jupyterlab-lsp 5.0.1

• bug fixes:

  • fix false “undefined name” in %%time and %%capture magics #1007 (thanks @i-aki-y!)

  • fix completion items for paths and other long items being cut off #1025

  • workaround issue with markdown lost on edit #1016

  • fix latex/Greek letters insertion and other completions which do not match prefix (do not pre-filter completions from kernel) #1022

  • fix completions in Console #1023

  • fix customising priority after pre-setting it with overrides.json #1027

  • fix jump to definitions in a file inside root in Pyright on Windows #1024

  • fix typos in setting title and help message #999 and #1010

• maintenance:

  • fix bootstrap script #1021

  • bump axios from 1.2.1 to 1.6.2 #1019

  • bump @babel/traverse from 7.22.5 to 7.23.4 #1020

jupyter-lsp 2.2.1

• bug fixes:

  • use APIHandler over JupyterHandler to get CORS OPTIONS #952 (thanks @zhanba!)

  • use shutil.which to discover npm executable on Windows (thanks @jameshurst!)

• maintenance:

  • resolve traitlets type warnings, lint, remove six #1015

  • use sys.executable in stdio tests #998

@jupyter-lsp/jupyterlab-lsp 5.0.0

• enhancements:

  • uses toast notifications instead of messages on status bar

  • diagnostics panel will be re-opened on reload

• maintenance:

  • support JupyterLab 4

  • use upstream @jupyterlab/lsp package

  • use camelCase convention in TypeScript/JavaScript code

  • use @codemirror/linter to show diagnostics

    • this comes with a different style of underlines and custom tooltips

• known issues/limitations:

  • configuration of language servers via JSON Settings Editor may result in a spurious warning due to a transitive clash with settings from the UI editor when using nested pattern (e.g. {pylsp: {flake8: {enabled: true}}}); the dotted pattern (e.g. {"pylsp.flake8.enabled": true}) does not lead to such problem.

  • enabling auto-invoke of completer requires toggling checkbox in both native and LSP Code Completion settings

  • robot mode does not support JupyterLab 4.0, hence robot LSP will not work either

  • renaming in docker files may not work on certain variables due to upstream tokenizer issue

Requires JupyterLab >=4.0.6,<5.0.0a0

@jupyter-lsp/jupyterlab-lsp 5.0.0-rc.1

• restore re-use of unused standalone connections

@jupyter-lsp/jupyterlab-lsp 5.0.0-rc.0

• fixes diagnostics not showing up in text editor in certain circumstances

• fixes signature hover box not being clickable

• improves performance by not creating a temporary editor to setup linter underline style

• JSON overrides work again

• fixes issue with syntax highlighting breaking when pasting multiple cells

Requires JupyterLab >=4.0.6,<5.0.0a0

@jupyter-lsp/jupyterlab-lsp 5.0.0-beta.1

• fix highlights conflict with selection

• fix scrolling to diagnostics and diagnostic rendering in windowed notebook

• suppress kernel completer in transclusions

@jupyter-lsp/jupyterlab-lsp 5.0.0-beta.0

• fix most regressions caught by tests in alpha

• reopen diagnostics panel on reload

• use toasts for notifications

@jupyter-lsp/jupyterlab-lsp 5.0.0-alpha.0

• maintenance:

  • support JupyterLab 4

Requires JupyterLab >=4.0.5,<5.0.0a0

@jupyter-lsp/jupyterlab-lsp 4.3.0 (2023-09-21)

• features:

• extractor for spark SQL magic was added (#980, thanks @skbitsp)

• classes needed to register custom adapters are now exported (#980, thanks @skbitsp)

  • using these classes is not recommended for new code as registration mechanism was rewritten in JupyterLab 4.0 and moved to @jupyterlab/lsp

@jupyter-lsp/jupyterlab-lsp 4.2.0 (2023-05-28)

• features:

  • diagnostics can be ignored by severity level with new ignoreSeverities setting (#940)

jupyter-lsp 2.2.0 (2023-05-28)

• bug fixes:

  • await sleep() coroutine (#939, thanks @jinzhen-lin)

  • limit attempts to initialise shadow file system to three trials (#936)

• maintenance:

  • support bash-language-sever 4.3.2+ (#938, thanks @ackalker)

@jupyter-lsp/jupyterlab-lsp 4.1.0 (2023-04-24)

• features:

  • make .virtual_documents optional - logic for solving URIs will now check if the server does not require it (#930)

• bug fixes:

  • fix contrast of mark highlights in dark mode (#928)

  • fix conflict with block/rectangular selections on Alt + click (#927)

• maintenance:

  • update texlab expectation (#929)

jupyter-lsp 2.1.0 (2023-04-24)

• features:

  • make .virtual_documents optional - added requires_documents_on_disk spec field (default True, will become False in next major version) (#930)

@jupyter-lsp/jupyterlab-lsp 4.0.1 (2023-03-21)

• bug fixes

  • overrides from overrides.json are now picked up again (#919)

jupyter-lsp 2.0.1 (2023-03-21)

• bug fixes:

  • pyright schema now includes required python. prefixes (#919)

@jupyter-lsp/jupyterlab-lsp 4.0.0 (2023-03-15)

Requires JupyterLab >=3.6.0,<4.0.0a0 and Python 3.8 or newer.

• features:

  • implement jump target selector and jump to references ([#739])

  • implement settings UI using native JupyterLab 3.3 UI (#778)

  • add option to show hover tooltip automatically (#864, thanks @yamaton)

  • implement eliding for long paths/files in completer (#893)

  • allow to pass default workspace config (serverSettings) in specs (#856)

• bug fixes:

  • use correct websocket URL if configured as different from base URL (#820, thanks @MikeSem)

  • clean up all completer styles when completer feature is disabled (#829).

  • fix undefined being inserted for path-like completion items with no insertText (#833)

  • reduce signature flickering when typing and hover flicker when moving mouse (#836)

  • fix sporadic misplacement of hover tooltips (#860, thanks @yamaton)

  • fix hover tooltip not updated after character deletions (#867, thanks @yamaton)

  • handle potential race condition in feature settings loading (#882)

• refactoring:

  • changed NPM packages namespace from @krassowski to @jupyter-lsp ([#862])

  • move client capabilities to features (#738)

• downstreams:

  • use the host application’s HTTP settings for requests to the REST and WebSocket routes ([#881])

  • source maps are provided for improved debugging (#882)

  • the derived JSON Schema types are avilable as SCHEMA (#882)

• documentation:

  • add missing --channel to conda instruction for texlab (#789, thanks @justin-f-perez)

  • remove references to pylsp fork (#800 and #814, thanks @joelostblom and @nickfong)

  • add Robot Framework language server (#724, thanks @bollwyvl)

  • add a list of third-party and community language servers (#826, thanks @cccs-jc)

  • fix documentation of .lsp_symlink workaround ([#828])

• maintenance:

  • bump minimum required JupyterLab version to 3.3 (>=3.3.0,<4.0.0a0)

  • bump minimum required Node.js version to 14.0 (12.0 reached EOL in April)

  • use newer @jupyterlab/builder which provides third-party license information (#882)

lsp-ws-connection 0.7.1 (2023-03-15)

• downstreams:

  • includes sourcemaps for easier debugging (#882)

• maintenance:

  • handle notification promises more explicitly (#882)

jupyter-lsp 2.0.0 (2023-03-15)

• dependencies:

  • uses importlib_metadata (or importlib.metadata on 3.10+) for entry_points (#882)

  • supports Python versions are 3.8 or newer

• performance:

  • entry_point discovery is deferred until server has started, improving jupyter_server startup time (#852)

• allow to pass default workspace config (serverSettings) in specs (#856)

@krassowski/jupyterlab-lsp 3.10.1 (2022-03-21)

• bug fixes:

  • fix navigation to files with names including characters which require encoding (#758)

  • fix omissions in the main settings schema for correct rendering in JupyterLab 3.3 Settings Editor UI (#779)

• maintenance

  • make tests pass with new JupyterLab 3.3 (#777)

  • pin r-base version to fix Binder (#766)

  • pin node-gyp to fix CI on Windows (#772)

  • pin pytest-flake8 to fix CI

  • update development/testing environment dependencies: urijs, typedoc, url-parse

@krassowski/jupyterlab-lsp 3.10.0 (2022-01-01)

• features:

  • enable pre-filtering of completion matches by default (#735)

  • add support for diagnostic tags: Deprecated, Unnecessary (#736, #737)

• bug fixes:

  • squash warnings and errors in web console (#732)

  • fix signature blur and fix formatting when no arguments are present (#734)

  • fixed with enabling of strict null checks:

    • previously changing kernels always led to restarting of LSP connection, even if the kernel language did not change; now the connection will be retained if possible

    • markdownRenderer is no longer implicitly required

    • diagnostics sorting with missing values for source and severity was improved and missing values will be consistently shown at the end

    • diagnostics placeholder was split into Diagnostics are not available and No issues detected, great job! which will now show up properly

• maintenance:

  • enable strict null checks and other strict settings (#733)

  • specify client capabilities in features instead of hard-coding them (#738)

  • bump minimum required JupyterLab version to 3.1 (>=3.1.0,<4.0.0a0)

@krassowski/code-jumpers 1.2.0 (2022-01-01)

• maintenance:

  • updates to API with respect to null/undefined values in course strict null checks activation (#733)

@krassowski/jupyterlab-lsp 3.9.3 (2021-12-19)

• bug fixes:

  • workaround upstream issue in merging translation (#726, thanks @fcollonval)

@krassowski/jupyterlab-lsp 3.9.2 (2021-12-12)

• bug fixes:

  • prevent very long completion details text from extending the completer indefinitely (#698)

  • correct status translations (#700, thanks @fcollonval)

  • fix translations in status pop-up (#703)

  • workaround issue causing file rename when opening external files by jumping to them (#712, thanks @jepcor97)

  • fix sorting by “Line:Ch” and “Cell” in Diagnostics Panel (#717)

  • fix header border missing when scrolling in Diagnostics Panel (#717)

• documentation improvements:

  • clarify that JupyterLab restart is needed after installation (#714, thanks @3coins)

@krassowski/completion-theme 3.2.0 (2021-12-12)

• features:

  • add details-below layout allowing to change the completer arrangement (#698)

jupyter-lsp 1.5.1 (2021-12-12)

• documentation improvements:

  • document troubleshooting steps for texlab server(#702)

• maintenance and upkeep:

  • migrate test configuration to ServerApp as needed (#713)

  • address deprecation warnings (#713)

@krassowski/jupyterlab-lsp 3.9.1 (2021-10-24)

• bug fixes:

  • fix build issues making it difficult to reuse/extend the package (#696, thanks @cccs-jc)

jupyter-lsp 1.5.0 (2021-10-24):

• features:

  • add support for new typescript-language-server replacing javascript-typescript-langserver; despite the name both packages provide support for all four: JavaScript, JSX, TypeScript and TSX; the old javascript-typescript-langserver can still be used, but it is no longer maintained and we will not be supported, and specs may be removed in the next major release (#697).

@krassowski/jupyterlab-lsp 3.9.0 (2021-10-17)

• features:

  • signature help box will now persist while typing the arguments of a function (#671)

  • the currently active argument will be highlighted in the signature help box

  • if the documentation exceeds a user-configurable number of lines the signature help box will only display the first line of the documentation and the following lines will be collapsed into an expandable details section.

  • the signature box is now displayed above the current line

  • the signature box takes up less space

  • you can now disable specific features in settings (#689)

• bug fixes:

  • fix missing translation strings (#675)

  • fix kernel completions not showing up for R (#686)

  • fix tab completions not showing up in strings due to incomplete trigger kind invalidation (#686)

  • fix path completions reconciliation for pyls/pylsp with IPython (#686)

  • improve escaping rule for IPython magics overrides (#688)

  • fix documentation panel not showing up when typing fast (#692)

@krassowski/jupyterlab-lsp 3.8.1 (2021-08-02)

• bug fixes:

  • %Rdevice magic is now properly overridden and won’t be extracted to R code (#646)

  • Fix hover rendering for MarkedStrings, fix hover disappearing when moving mouse towards it (#653)

jupyter-lsp 1.4.1 (2021-08-02)

• bug fixes:

  • remove spurious ValidationError warnings for non-installed servers (#645, thanks @karlaspuldaro)

  • reduce number and verbosity of logs on startup

@krassowski/jupyterlab-lsp 3.8.0 (2021-07-04)

• improvements:

  • add a note on manually enabling backend extension (#621, thanks @icankeep)

  • in-app troubleshooting/installation help is now offered for servers which are needed but could not be detected (if auto-detection specification for those is present) (#634)

• bug fixes:

  • fix rename shortcut registration in file editor (#614)

  • improve performance of icon rendering in completer (#625)

  • fix cache misses and concurrency issues for hover (#630)

jupyter-lsp 1.4.0 (2021-07-04)

• features:

  • troubleshoot property was added to the language server spec, allowing to describe auto-detection troubleshooting (#634)

  • new endpoint specs will list all language server specifications known to jupyter-lsp allowing frontends to suggest installation of specific language servers (#634)

• changes:

  • ShellSpec.is_installed() signature changed; it now accepts the LanguageServerManagerAPI rather than the resolved command location (of str type); the specs using only is_installed_args are not affected; as this method was only used by internally by the __call__ implementation (which was adjusted accordingly) this change should not break existing specs unless any of these methods were overridden in sub-classes.

  • SpecBase was moved to types.py; it can still be imported from utils, but doing so is discouraged

  • ShellSpec.solve() was added to facilitate discovery of command location

jupyter-lsp 1.3.0 (2021-06-02)

• features:

  • add auto-detection of pyright server (#587, thanks @yuntan)

@krassowski/jupyterlab-lsp 3.7.0 (2021-05-31)

• features:

  • add ability to deactivate Kernel completions or LSP completion through the settings (#586, thanks @Carreau)

  • allow to set a priority for LSP server, allowing to choose which server to use when multiple servers are installed (#588)

  • add auto-detection of pyright server (#587, thanks @yuntan)

  • update from JupyterLab Classic to RetroLab (#602)

  • log server messages in user-accessible console (#606)

• bug fixes:

  • workaround url-parse issue causing problems when using JupyterLab 3.0.15 (#599)

• other changes:

  • drop Node 10 (EOL 2 weeks ago) testing on CI, add Node 15 (#587)

  • update lsp-ws-connection dependencies (#606)

  • old emit-based API of lsp-ws-connection is new deprecated and will be removed in the next major version; please use serverNotifications, clientNotifications, clientRequests and serverRequests instead (#606)

jupyter-lsp 1.2.0 (2021-04-26)

• features:

  • add support for python-lsp-server, a community fork of pyls (#584 and #582, thanks @maresb)

• bug fixes:

  • fix encoding on Windows for non-conda installs (#580, thanks @stonebig)

@krassowski/code-jumpers 1.1.0 (2021-04-26)

• features:

  • added translation support (#557, thanks @JessicaBarh)

@krassowski/jupyterlab-lsp 3.6.0 (2021-04-26)

• features:

  • added translation support (#557, thanks @JessicaBarh)

• bug fixes:

  • fixed name of jupyterlab-lsp package displayed in JupyterLab UI (#570, thanks @marimeireles)

  • removed vendored CodeMirror from distribution (#576)

@krassowski/jupyterlab-lsp 3.5.0 (2021-03-22)

• features:

  • adds %%bigquery IPython cell magic support for BigQuery (#553, thanks @julioyildo)

  • completions filtering can be set to case-insensitive in settings (#549)

  • completions filtering can hide exact matches (#549)

  • the extra information displayed next to the completion label now can include ‘detail’ (usually module/package of origin), and can be customized in settings (#549)

• bug fixes:

  • prevents throwing a highlights error when adding new cell with Shift + Enter (#544)

  • fixes IPython pinfo and pinfo2 (? and ??) for identifiers containing s (#547)

  • fixes incorrect behaviour of LSP features in some IPython magics with single line of content (#560)

  • fixes name of jupyterlab-lsp package in JupyterLab

• for extension authors:

  • minimal functional extractor and code overrides APIs are now exported; these APIs cab be subject to change in future releases (#562)

jupyter-lsp 1.1.4 (2021-02-21)

• bug fixes:

  • ensures jupyter*_config.d paths are searched for language_servers as documented in Configuring (#535)

  • uses more explicit file name for enabling jupyter-lsp in notebook and `jupyter_server (#535)

@krassowski/jupyterlab-lsp 3.4.1 (2021-02-16)

• bug fixes:

  • fixed installation of the source version of the extension (causing build error if classic was not installed) (#526)

@krassowski/jupyterlab-lsp 3.4.0 (2021-02-14)

• features:

  • the priority of the completions from kernel can now be changed by switching new kernelCompletionsFirst setting (#520)

  • completer panel will now always render markdown documentation if available (#520)

    • the implementation re-renders the panel as it is the best we can do until jupyterlab#9663 is merged

  • the completer now uses filterText and sortText if available to better filter and sort completions (#520, #523)

  • completer suppressInvokeIn setting was removed; suppressContinuousHintingIn and suppressTriggerCharacterIn settings were added (#521)

  • suppressContinuousHintingIn by default includes def to improve the experience when writing function names (#521)

  • long file paths are now collapsed if composed of more than two segments to avoid status popover and diagnostics panel getting too wide (#524)

• bug fixes:

  • user-invoked completion in strings works again (#521)

  • completer documentation will now consistently show up after filtering the completion items (#520)

  • completions containing HTML-like syntax will be displayed properly (an upstream issue) (#520, #523)

  • diagnostics panel will no longer break when foreign documents (e.g. %%R cell magics) are removed (#522)

@krassowski/jupyterlab-lsp 3.3.1 (2021-02-07)

• bug fixes:

  • completion and signature suggestions get invalidated when editor changes (#507)

  • signature suggestions now invalidate on cursor move to another line or backwards too (#507)

  • LaTeX is now rendered in documentation panel of completer (#506)

  • completion response returned as plain text use pre tag to retain whitespace formatting (#506)

  • pre-formatted code font size was reduced to match font-size of the text in completion panel (#506)

  • completer no longer spans the entire screen if it has long entries (#506)

jupyter-lsp 1.1.3 (2020-02-07)

• features:

  • add config for the classic notebook server extension (#504)

@krassowski/jupyterlab-lsp 3.3.0 (2021-01-31)

• features:

  • added a timeout for kernel completion, with the default of 600ms (#496)

  • added an option to skip waiting for kernel completions if busy, off by default (#496)

• bug fixes:

  • delayed completion suggestions will no longer show up if cursor moved to another line (#496)

  • changes in notebooks after kernel restart or file rename will now be recorded by the language server again (#496)

  • when either of kernel providers: kernel or LSP server fails, the completion from the other will still be shown (#496)

jupyter-lsp 1.1.2 (2021-01-31)

• bug fixes:

  • fixed issues with language server messages being truncated in certain circumstances on Windows

@krassowski/jupyterlab-lsp 3.2.0 (2021-01-24)

• features:

  • documentation panel in completer now works for R language too: implemented completionItem/resolve (#487)

  • kernel types returned by IPython and IJulia are now mapped to LSP types; you can customize the mappings in settings (#487)

• bug fixes:

  • diagnostics panel works after kernel restart properly (#485)

  • workaround was added to enable jedi-language-server diagnostics (#485)

  • Julia language server will not crash when saving a non-Julia file: fixed sendSaved notification scope (#491)

jupyter-lsp 1.1.1 (2021-01-24)

• bug fixes:

  • PythonModuleSpec no longer raises exception when the server module does not exist (#485)

@krassowski/jupyterlab-lsp 3.1.0 (2021-01-17)

• features

  • make the extension work with jupyterlab-classic - experimental, not all features are functional yet (#465)

  • new status “Server extension missing” and a dialog with advice was added to help users with atypical configurations (#476)

  • for developers: the verbosity of console logs is now controllable from settings and set to warn by default (#480)

• bug fixes:

  • namespace completions in R (after double and triple colon) now work properly (#449)

  • improved status icon contrast when status item is active (#465)

  • connection manager now properly keeps track of notebooks when multiple notebooks are open (#474)

  • new cells added after kernel restart now work properly; kernel changes are handled correctly (#478)

  • increase total timeout for language server connection (#479)

  • fix status communication during initialization (#479)

jupyter-lsp 1.1.0 (2021-01-17)

• features

  • added experimental detection of Julia and Jedi language servers (#481)

• bug fixes:

  • send periodic pings on websocket channels to maintain connection (#459, thanks @franckchen)

  • R languageserver is no longer incorrectly shown as available when not installed (#463)

  • fix completion of very large namespaces (e.g. in R’s base or in JavaScript) due to truncated message relay (#477)

@krassowski/jupyterlab-lsp 3.0.0 (2021-01-06)

• features

  • support for JupyterLab 3.0 (#452, #402)

jupyter-lsp 1.0.0 (2021-01-06)

• features

  • support for JupyterLab 3.0 (#452, #402)

@krassowski/jupyterlab-lsp 2.1.2 (2021-01-02)

• features

  • highlights can now be auto-removed from the cells/editors on blur (set removeOnBlur to true in settings) (#446)

• bug fixes

  • improved performance of completion and highlights by minimising the number of highlight requests and GUI redraws (token checking, debouncing, acting on a single response only) (#433)

  • highlights now update after cell focus/blur events even if those do not trigger cursor movement (#433)

  • trigger characters auto-invoke now works in continuous hinting mode again (#434)

@krassowski/jupyterlab-lsp 2.1.1 (2020-12-15)

• bug fixes

  • fix crash “blank screen” caused by Mac command character included in jump-to schema file (#429)

jupyter-lsp 0.9.3 (2020-12-13)

• features

  • the virtual documents’ folder can be configured with JP_LSP_VIRTUAL_DIR or LanguageServerManager.virtual_documents_dir, with a potential benefit for JupyterHub installations (the default value remains contents.root_dir / .virtual_documents) (#416, thanks @fcollonval)

@krassowski/jupyterlab-lsp 2.1.0 (2020-12-13)

• features

  • added “click to jump” functionality (by default using Alt modifier) (#423)

  • added “jump back” command, by default activated with Alt + o (#423)

  • .virtual_documents location can now be customized (#416)

  • tokens are now exported making them available for import from other extensions (#414, thanks @martinRenou)

• bug fixes

  • context menu commands are now correctly registered where previously specific conditions were leading to race conditions (#399, thanks @mnowacki-b)

  • handles characters that need escaping (spaces, non-ASCII characters) more robustly in files and folder names (#403, thanks @bollwyvl and @avaissi)

  • moving cells now triggers the document update immediately leading to immediate diagnostics update (#421)

  • changing cell type to raw or markdown and then back to code properly unbinds/binds event handlers and updates document (#421)

  • pasted cells are added to the LSP document immediately, without the need for the user to enter them (#421)

  • improved error message when language server cannot be found (#413, thanks @martinRenou)

  • developer documentation got improved (#412, thanks @karlaspuldaro)

@krassowski/code-jumpers 1.0.0 (2020-12-13)

• breaking changes

  • split away from @krassowski/jupyterlab_go_to_definition, renamed to @krassowski/code-jumpers (#423):

    • removed unused code

    • refactored history operations to track files and always use global location data

  • renamed uri to contents_path to help avoid programmer issues with characters requiring URI encoding (#406)

@krassowski/jupyterlab-lsp 2.0.8 (2020-10-25)

• bug fixes

  • custom cell syntax highlighting is now properly removed when no longer needed (#387)

  • the completer in continuous hinting now works well with the pasted text (#389)

  • continuous hinting suggestions will no longer show up if the only hint is the same as the current token (#391)

  • available options for hover modifier keys are now listed in the setting descriptions (#377)

@krassowski/jupyterlab-lsp 2.0.7 (2020-09-18)

• bug fixes

  • fix syntax highlighting in hover tooltips and reduce unnecessary padding and margin (#363)

  • greatly improve performance of hover action (#363)

  • improve support for expanded hovers tooltips using deprecated API (#363)

  • do not hide hover tooltips too eagerly (allowing selecting text/easy scrolling of longer tooltips) (#363)

@krassowski/jupyterlab-lsp 2.0.6 (2020-09-15)

• bug fixes

  • fix syntax highlighting of %%language cells slowing down editing in notebooks (#361)

@krassowski/jupyterlab-lsp 2.0.5 (2020-09-11)

• bug fixes

  • fix too aggressive overrides of IPython’s pinfo (?) and pinfo2 (??) (#352)

@krassowski/jupyterlab-lsp 2.0.4 (2020-09-11)

• bug fixes

  • improve code overrides for IPython line magics (#346)

  • implement missing code overrides for IPython’s pinfo (?) and pinfo2 (??) syntactic sugar (#346)

@krassowski/jupyterlab-lsp 2.0.2 (2020-09-07)

• bug fixes

  • fix code overrides not being registered properly (#340)

@krassowski/jupyterlab-lsp 2.0.1 (2020-09-07)

• bug fixes

  • bump version of lsp-ws-connection dependency to fix the LaTeX server issues (see #337)

jupyter-lsp 0.9.2 (2020-09-03)

• autodetects the sql language server for .sql files (#328)

  • diagnostics are provided by sqlint which requires Node 11+ to work well (in contrast to currently required Node 10+).

@krassowski/jupyterlab-lsp 2.0.0 (2020-09-03)

• features

  • support for JupyterLab 2.2 (#301)

  • completer now displays server-provided documentation, and a kernel icon for kernel suggestions without type information (#301)

  • add two icons themes for the completer (material and vscode) (#322)

  • the documentation by the completer can be turned on or off (#315)

  • continuous hinting (Hinterland mode) can be enabled in settings (#315)

  • tokens in which the completer should not be triggered can be changed (#315)

  • configuration for the following features is now exposed in the settings editor (#318):

    • diagnostics (display, filtering)

    • hover (modifier key)

  • rename operation status reporting got improved (#318)

  • replaced the generic status icons with code check icon (coloured differently according to the status) (#318)

  • added icons for all the features and their commands (#318)

  • refactored the codebase with a new architecture which allows dynamic features, document widget adapter, and code editor registration (#318)

  • the document in the connections list in the statusbar popover are now represented by easy-to-understand DocumentLocator (breadcrumbs) widget rather than an internal id (bacc006)

  • syntax highlighting mode is adjusted to the language with the majority of the code in an editor (#319)

  • copy diagnostics message and filter diagnostics from context menu of Diagnostic Panel (#330)

• bug fixes

  • path-autocompletion issues were resolved upstream and this release adopts these changes

  • the missing caret and document connection icons were restored in the statusbar popover (#318)

  • pressing “Cancel” rename during rename now correctly aborts the rename operation (#318)

  • when a language server for a foreign document is not available an explanation is displayed (rather than the “Connecting…” status as before) (4e5b2ad)

  • when jump target is not found a message is now shown instead of raising an exception (00448d0)

  • fixed status message expiration and replacement (8798f2d), (#329)

  • fixed some context command rank issues introduced after an attempt of migration to nulls (#318)

@krassowski/jupyterlab-lsp 1.1.2 (2020-08-05)

• bug fixes

  • emits console warnings instead of throwing errors in hover handlers and connections (#299, #300)

  • improve URL checks in message handling to enable LaTeX diagnostics to work when configured (#288)

jupyter-lsp 0.9.1 (2020-08-05)

• autodetects the texlab language server for .tex files (#288)

  • diagnostics should be provided by chktex on save, but don’t yet appear, but can be configured through the Advanced Settings Editor to appear on save or change

@krassowski/jupyterlab-lsp 1.1.0 (2020-07-20)

• features

  • language servers can now be configured from the Advanced Settings Editor (#245)

• bug fixes

  • fixes currently-highlighted token in dark editor themes against light lab theme (and vice versa) (#195)

  • restores sorting order-indicating caret icons in diagnostics panel table (#261)

  • handles document open and change operation ordering more predictably (#284)

  • fixes some pyflakes issues caused by line magics substitution (#293)

  • updated the link to the documentation of language servers (#294)

jupyter-lsp 0.9.0 (2020-07-20)

• features

  • language servers can now be configured from the Advanced Settings Editor (#245)

• bug fixes

  • handles document open and change operation ordering more predictably (#284)

lsp-ws-connection 0.5.0 (2020-07-20)

• features

  • language servers can now be configured from the Advanced Settings Editor (#245)

• bug fixes

  • handles document open and change operation ordering more predictably (#284)

@krassowski/jupyterlab-lsp 1.0.0 (2020-03-14)

• features

  • supports JupyterLab 2.0

@krassowski/jupyterlab_go_to_definition 1.0.0 (2020-03-14)

• features

  • supports JupyterLab 2.0

@krassowski/jupyterlab-lsp 0.8.0 (2020-03-12)

• features

  • opens a maximum of one WebSocket per language server (#165, #199)

  • lazy-loads language server protocol machinery (#165)

  • waits much longer for slow-starting language servers (#165)

  • cleans up documents, handlers, events, and signals more aggressively (#165)

  • ignores malformed diagnostic ranges, enabling markdown support (#165)

  • passes tests on Python 3.8 on Windows (#165)

  • improves support for rpy2 magic cells with parameters ( #206 )

• bug fixes

  • reports files are open only after installing all handlers to avoid missing messages (#201)

lsp-ws-connection 0.4.0 (2020-03-12)

• breaking changes

  • no longer assumes one document per connection (#165)

  • requires documents be opened explicitly (#165)

  • use of the eventEmitter pattern mostly deprecated in favor of Promises (#165)

jupyter-lsp 0.8.0 (2020-03-12)

• breaking changes

  • websockets are now serviced by implementation key, rather than language under lsp/ws/<server key> (#199)

  • introduces schema version 2, reporting status by server at lsp/status (#199)

• bugfixes:

  • handles language server reading/writing and shadow file operations in threads (#199)

jupyter-lsp 0.7.0

• bugfixes

  • didSave no longer causes unwanted messages in logs ( #187 )

@krassowski/jupyterlab-lsp 0.7.1

• features

  • users can now choose which columns to display in the diagnostic panel, using a context menu action ( #159 )

  • start the diagnostics panel docked at the bottom and improve the re-spawning of the diagnostics panel ( #166 )

• bugfixes

  • fixed various small bugs in the completer ( #162 )

  • fix documentation display in signature for LSP servers which return MarkupContent ( #164 )

lsp-ws-connection 0.3.1

• added sendSaved() method (textDocument/didSave) ( #147 )

• fixed getSignatureHelp() off-by-one error ( #140 )

@krassowski/jupyterlab-lsp 0.7.0

• features

  • reduced space taken up by the statusbar indicator ( #106 )

  • implemented statusbar popover with connections statuses ( #106 )

  • generates types for server data responses from JSON schema ( #110 )

  • added ‘rename’ function for notebooks, using shadow filesystem ( #115 )

  • added a UX workaround for rope rename issues when there is a SyntaxError in the Python code ( #127 )

  • added a widget panel with diagnostics (inspections), allowing to sort and explore diagnostics, and to go to the respective location in code (with a click); accessible from the context menu ( #129 )

  • all commands are now accessible from the command palette ( #142 )

  • bash LSP now also covers %%bash magic cell in addition to %%sh ( #144 )

  • rpy2 magics received enhanced support for argument parsing in both parent Python document (re-written overrides) and exctracted R documents (improved foreign code extractor) ( #148, #153 )

  • console logs can now easily be redirected to a floating console windows for debugging of the browser tests (see CONTRIBUTING.md)

• bugfixes

  • diagnostics in foreign documents are now correctly updated ( 133fd3d )

  • diagnostics are now always correctly displayed in the document they were intended for

  • the workaround for relative root path is now also applied on Mac ( #139 )

  • fixed LSP of R in Python (%%R magic cell from rpy2) ( #144 )

  • completion now work properly when the kernel is shut down ( #146 )

  • a lowercase completion option selected from an uppercase token will now correctly substitute the incomplete token ( #143 )

  • didSave() is emitted on file save, enabling the workaround used by R language server to lazily load library(tidyverse) ( #95, #147, )

  • signature feature is now correctly working in notebooks ( #140 )

lsp-ws-connection 0.3.0

• infrastructure

  • brought into monorepo #107

• dev

  • allows initializeParams to be overloaded by subclasses

  • adopts

    • typescript 3.7

    • prettier

    • tslint

  • added initialization checks before executing sendChange to comply with LSP specs #115

jupyter-lsp 0.7.0b0

• features

  • adds a language server status endpoint ( #81 )

  • adds more descriptive information to the language server spec ( #90 )

  • adds an extensible listener API ( #99, #100 )

@krassowski/jupyterlab-lsp 0.6.1

• features

  • adds an indicator to the statusbar

  • and many other improvements, see the release notes

• dependencies

  • removes unused npm dependencies

@krassowski/jupyterlab-lsp 0.6.0

• features

  • allows “rename” action in file editor

• bugfixes

  • handles some non-standard diagnostic responses

• testing

  • adds browser-based testing for file editor

• dependencies

  • requires jupyter-lsp

jupyter-lsp 0.6.0b0

• features

  • starts language servers on demand

  • accepts configuration via Jupyter config system (traitlets) and python entry_points

  • autodetects language servers for bash, CSS, LESS, SASS, Dockerfile, YAML, JS, TypeScript, JSX, TSX, JSON, YAML

• bugfixes

  • fix issue that variables declared in cell magics(%%time, %%capture) are masked( #635 )

Roadmap

If a feature you would like is not on the lists above, please feel free to suggest it by opening a new issue.

Frontend

If you would like to help with any item please open an issue to discuss (or directly open pull request if it is a small enhancement)

• settings enhancements, including:

  • options to change the verbosity of signature documentation hints (number of lines to be shown)

  • custom foreign extractors allowing to customize behaviour for magics

• code actions (allowing to “quick fix” a typo, etc.)

• code formatting

• gutter with linter results

• use the kernel session for autocompletion in FileEditor if available

Backend

• #49 cookiecutter for pip-installable specs

• add hook system to allow serverextensions/kernels to modify, inspect and react to LSP messages

Architecture

As-Is

These are how we think everything works in the current main branch.

Front End

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], line 1
----> 1 get_ipython().run_line_magic('dot', 'dot/as-is/frontend.dot')

File ~/checkouts/readthedocs.org/user_builds/jupyterlab-lsp/conda/v5.0.3/lib/python3.10/site-packages/IPython/core/interactiveshell.py:2456, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth)
   2454     kwargs['local_ns'] = self.get_local_scope(stack_depth)
   2455 with self.builtin_trap:
-> 2456     result = fn(*args, **kwargs)
   2458 # The code below prevents the output from being displayed
   2459 # when using magics with decorator @output_can_be_silenced
   2460 # when the last Python token in the expression is a ';'.
   2461 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):

Cell In[1], line 10, in dot(line, cell)
      8 @IPython.core.magic.register_line_cell_magic
      9 def dot(line, cell=None):
---> 10     src = graphviz.Source(pathlib.Path(line).read_text() if line else cell)._repr_svg_()
     11     src = re.sub(r"<svg (.*)viewBox", "<svg viewBox", src, flags=re.M | re.DOTALL)
     12     return IPython.display.SVG(data=src)

AttributeError: 'Source' object has no attribute '_repr_svg_'

Positioning system

Back End

Proposals

Some fragments of how the architecture could change in the future, and why (or why not) they might be a good idea.

• Reorganize client source with lerna and typescript projects #76

• Add DiagnosticsManager, refactor DiagnosticPanel #176

• Multiple sources of LSP messages on frontend and backend #184

• Formalize and extend language transclusion #191