View IPython notebooks in GitLab

gitlab-ipython-notebook.md

Viewing IPython notebooks in GitLab

GitLab is open source software to collaborate on code (a GitHub clone to run on your own server). Clicking a blob (a file in a repository) in GitLab shows a nice rendering if GitLab supports the file type (e.g., images, Markdown documents), or its content as plain text otherwise. The patch described here adds support to GitLab for rendering IPython notebooks (.ipynb files).

Obligatory warning

This is a Quick Hack (tm):

• It is slow
• It probably has some security issue

If you run this on a public production server, your cat might get electrocuted. Other than that, it should be fine (not the cat).

Requirements

On the GitLab server:

• Pandoc - apt-get install pandoc
• IPython (>= 1.0) and Jinja2 - pip install ipython jinja2

Installation

Note: The following instructions are for GitLab 8.10. Whenever a new version comes out, I'll try to have a branch ready for that version so you can just substitute the current version for 8-10-stable below.

New GitLab installation

Install GitLab as you normally would with the following changes:

• Clone the GitLab source code from https://github.com/martijnvermaat/gitlabhq.git (instead of https://github.com/gitlabhq/gitlabhq.git)
• Checkout the 8-10-stable-blob-ipython-notebook branch (instead of 8-10-stable)

Existing GitLab installation

If you already have a GitLab installation running, follow the usual update guide with the following commands instead of the Get latest code step:

cd /home/git/gitlab
sudo -u git -H git remote add martijnvermaat https://github.com/martijnvermaat/gitlabhq.git
sudo -u git -H git fetch martijnvermaat
sudo -u git -H git checkout 8-10-stable-blob-ipython-notebook

For developers and Aerosmith

Alternatively, if you know what you're doing or like living on the edge, use branch blob-ipython-notebook, which is based on master. You probably want to rebase on upstream master in that case too.

Configuration

If needed, all can be disabled by setting ipython_notebook.render to false in config/gitlab.yml.

In the same file, you can specify a custom command for converting notebook files with ipython_notebook.nbconvert. This command should accept an IPython notebook file as its first argument and write an HTML representation to standard output.

Implementation

The obvious implementation would be to link to or embed the IPython Notebook Viewer. However, GitLab is often used for private repositories where we obviously cannot share the content with a public service.

So this is an alternate implementation, where the notebook file is converted to HTML on the fly by the IPython nbconvert tool. The rendered notebook is returned verbatim to the browser and subject to the regular GitLab authentication rules. Rendered notebooks are not cached.

A previous implementation embedded the rendered notebook in the GitLab page as an IFrame using a data URI. Since this approach had some technical limitations and exposed some browser bugs with more complex notebooks, the current implementation renders the notebook outside the GitLab interface. The IFrame approach can still be found in branch blob-ipython-notebook-iframe (based on master shortly after the release of GitLab 7.3.2).

screenshot-iframe.png

screenshot.png

Hi Martijn;

I've been trying to install this in a Docker on top of sameersbn's docker-gitlab. So, I use his image, then follow your instructions for Existing GitLab installation. Then there's a little more magic to get the correct gitlab.yml (because I need all sameer's config changes, plus yours) file, but everything seems to be in place (the six files you've modified are all there), and when I browse to an .ipynb file, I see the extra green View button that wasn't in my omnibus install.

However, if I click on the green button, I get a "raw" display:

Started GET "/instrumentation/data_handling/ipython_notebook/master/Ipython_notebook_test.ipynb" for 192.171.163.11 at 2016-05-27 16:24:27 +0100
Processing by Projects::IpythonNotebookController#show as HTML
  Parameters: {"namespace_id"=>"instrumentation", "project_id"=>"data_handling", "id"=>"master/Ipython_notebook_test.ipynb"}
Redirected to http://gitserverdev.sahfos.dev/instrumentation/data_handling/raw/master/Ipython_notebook_test.ipynb
Completed 302 Found in 304ms (ActiveRecord: 6.5ms)
Started GET "/instrumentation/data_handling/raw/master/Ipython_notebook_test.ipynb" for 192.171.163.11 at 2016-05-27 16:24:27 +0100
Processing by Projects::RawController#show as HTML
  Parameters: {"namespace_id"=>"instrumentation", "project_id"=>"data_handling", "id"=>"master/Ipython_notebook_test.ipynb"}
Completed 200 OK in 91ms (ActiveRecord: 6.1ms)

It definitely finds a controller, but then reverts to the Raw handler. Unfortunately, I am completely unfamiliar with Ruby. Do you have any idea what I'm missing?

Hi @auspex, thanks for checking this out. I have to say I'm no Ruby programmer either, so don't really have an idea where this is going wrong.

Did you make sure to apply my patch on a corresponding GitLab version? E.g., if you use GitLab 8.6, use the patch in the 8-6-stable-blob-ipython-notebook branch?

We just deployed the GitLab 8.8 version of this in our organisation and it's working fine. We're not using Docker though.

Sorry, I'm not very familiar with gist and it didn't let me know there'd been a response.

Yes, I was using 8.7 for both the base Docker and your patches. I had to get them matching, as I needed to do a backup/restore of the database. Anyway, I'm trying to redo this in 8.8, now, so I'll see how it goes.

Hello Martijn
Great work with this fork, hopefully gitlab will incorportate it (or a similar solution) into their hosted copy. I've tried to install it, and for some reason it's just not working. I performed a installation from source, but cloning from your repository instead of the original one. After I am done with the installation, I can see a green button "view" in notebook files. When I click this button, another tab is openned but with the raw json, not the rendered version of the notebook. Any tips?

I created this gist to keep track of what was going on.

Best,Thiago

UPDATE -- Martijn, I just realized -- after going through the configs that nbconvert was not installed. However VERY obvious, maybe you can add a quick note about this.

Hi All,

I have a default installation of gitlab with apt-get (Ubuntu 16.04). Is there a fast way to incorporate/update it with this version in order to test the notebook viewer?

Best, George

Hi @martijnvermaat

Same question as George above. We have an Omnibus install using this command on Ubuntu 16.04:

dpkg -i gitlab-ce-XXX.deb

How can we make your awesome patch work for us?

Did you create this plugin because Gitlab does not natively render images/graphs in Jupyter notebooks?