Skip to content

Instantly share code, notes, and snippets.

@damianavila

damianavila/nikipy.ipynb

Last active August 29, 2015 14:03
Show Gist options
  • Save damianavila/5a4d290b9b78a36ac51d to your computer and use it in GitHub Desktop.
Save damianavila/5a4d290b9b78a36ac51d to your computer and use it in GitHub Desktop.
Download ZIP
A new bridge to share your content with the world: NikIPy.
Display the source blob
Display the rendered blob
Raw
nikipy.ipynb
{
"metadata": {
"name": "",
"signature": "sha256:7755067b60fcff54fd535494b70e36add3ca31c76a7777b20f221ce8617380b0"
},
"nbformat": 3,
"nbformat_minor": 0,
"worksheets": [
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# A new bridge to share your content with the world: *NikIPy*."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This poster nbviewered: http://nbviewer.ipython.org/gist/damianavila/5a4d290b9b78a36ac51d"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Abtract\n",
"\n",
"**Nikola** is a full-featured and extensible static website and blog generator.\n",
"It provides multiple core features such as: front pages, RSS feeds, tags, custom search, monthly/yearly archives, template customization, internationalization support, **Google** sitemap generation, custom deployment, light markups, image galleries... and much more, and also is easily extensible by plugins and themes.\n",
"\n",
"But one of the major **Nikola** features is the deep interaction with **IPython**, a comprehensive environment for interactive and exploratory computing. In fact, **Nikola** is able to render **IPython notebooks** into static HTML pages, making very easy to convert your notebook experiments, ideas and documentation into useful posts to share with the world.\n",
" And the interaction is deep enough to let us use the **IPython notebook** as a complete user interface to run all the Nikola functionalities. \n",
"\n",
"Besides there are some other static site generator able to render **IPython notebooks**, none of them support a bidirectional workflow with **IPython** as **Nikola** does. So, we do not longer have two apps, now we have a new entity: The **NikIPy Bridge**, a full featured platform which let us setup our website/blog in the cloud and be able to write our content from everywhere."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Nikola\n",
"\n",
"## What's Nikola and what can you do with it?\n",
"\n",
"**Nikola** is a static website and blog generator. The very short explanation is that it takes some texts you wrote, and uses them to create a folder full of HTML files. If you upload that folder to a server, you will have a rather full-featured website, done with little effort.\n",
"\n",
"It's original goal is to create blogs, but it supports most kind of sites, and can be used as a CMS, as long as what you present to the user is your own content instead of something the user generates.\n",
"\n",
"Keep in mind that \"static\" doesn't mean boring. You can have animations, slides or whatever fancy CSS/HTML5 thingie you like. It only means all that HTML is generated already before being uploaded. On the other hand, Nikola sites will tend to be content-heavy. What **Nikola** is good at is at putting what you write out there.\n",
"\n",
"## Features:\n",
"\n",
"If you want to create a blog or a site, **Nikola** provides:\n",
"\n",
"* Front page (and older posts pages)\n",
"* RSS Feeds\n",
"* Pages and feeds for each tag you used\n",
"* Custom search\n",
"* Full yearly archives\n",
"* Custom output paths for generated pages\n",
"* Easy page template customization\n",
"* Static pages (not part of the blog)\n",
"* Internationalization support (my own blog is English/Spanish)\n",
"* Google sitemap generation\n",
"* Custom deployment (if it's a command, you can use it)\n",
"* A(very) basic look and feel you can customize, and is even text-mode friendly\n",
"* The input format is light markup (reStructuredText or Markdown)\n",
"* Easy-to-create image galleries\n",
"* Support for displaying source code\n",
"* Image slideshows\n",
"* Client-side cloud tags\n",
"* A preview web server\n",
"* \"Live\" re-rendering while you edit\n",
"* \"Smart\" builds: only what changed gets rebuilt (usually in seconds)\n",
"* Easy to extend with minimal Python knowledge.\n",
"* And more, more and more...\n",
"\n",
"## Nikola ecosystem\n",
"\n",
"* **Nikola** itself: an open source project (MIT license) started around 2 years ago by Roberto Alsina. Currently we have more than 80 contributors and around 8-10 core devs.\n",
"\n",
"![](https://gist.githubusercontent.com/damianavila/41030242043536bb25ce/raw/d41812e13ac873c9f051037282395af923b4cc83/nikipy_1.png)\n",
"\n",
"* **Nikola** themes: we a several **mako** and **jinja** **booptrap**-based themes available and easily installable. And you can also write you own theme (I made my own, called **Zen themes** currently used in my blog).\n",
"\n",
"![](https://gist.githubusercontent.com/damianavila/41030242043536bb25ce/raw/c60394d937d3ea8758e8db1796f88ffe390f0ed0/nikipy_2.png)\n",
"\n",
"* **Nikola** plugins: we also provide, as a community, a lot of easilly installable plugins with useful features (but not enough general to be inside the core).\n",
"\n",
"![](https://gist.githubusercontent.com/damianavila/41030242043536bb25ce/raw/74bc621a13d36c47733970988aa6f857d5e69f44/nikipy_3.png)\n",
"\n",
"* **Nikola** users: we have a lot of users! and we want to have more!!! If you know someone using **Nikola** and not listed in the Nikola users site, let us know.\n",
"\n",
"![](https://gist.githubusercontent.com/damianavila/41030242043536bb25ce/raw/2323ec3925e8e4f14da454db560be59e70b4c071/nikipy_4.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# IPython\n",
"\n",
"**IPython** provides a rich architecture for interactive computing with:\n",
"\n",
"* Powerful interactive shells (terminal and Qt-based).\n",
"* A browser-based notebook with support for code, text, mathematical expressions, inline plots and other rich media.\n",
"* Support for interactive data visualization and use of GUI toolkits.\n",
"* Flexible, embeddable interpreters to load into your own projects.\n",
"* Easy to use, high performance tools for parallel computing.\n",
"\n",
"![](https://gist.githubusercontent.com/damianavila/41030242043536bb25ce/raw/831be4d66f4bcb43828c34395eadf521953fbc38/nikipy_5.png)\n",
"\n",
"You probably already know **IPython**, so let's go ahead!!\n",
"\n",
"**Note**: If you really don't know it, go to the **IPython** site, downlaod **IPython**, play with it and come back and read me again."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Nikola IPython Integration\n",
"\n",
"* Nikola supports `ipynbs` (IPython notebooks) in its core!\n",
"\n",
"You can write your posts (or site) in **ReST** and **Markdown** (or using other markups provided by **Nikola** plugins) but you can also write your content in **IPython** notebooks (using all the formatting available inside **IPython**) and it will be rendered as HTML inside the **Nikola** site context.\n",
"\n",
"You can see the implementation below:\n",
"\n",
"```python\n",
"\"\"\"Implementation of compile_html based on nbconvert.\"\"\"\n",
"\n",
"from __future__ import unicode_literals, print_function\n",
"import codecs\n",
"import os\n",
"\n",
"try:\n",
" from IPython.nbconvert.exporters import HTMLExporter\n",
" from IPython.nbformat import current as nbformat\n",
" from IPython.config import Config\n",
" flag = True\n",
"except ImportError:\n",
" flag = None\n",
"\n",
"from nikola.plugin_categories import PageCompiler\n",
"from nikola.utils import makedirs, req_missing\n",
"\n",
"\n",
"class CompileIPynb(PageCompiler):\n",
" \"\"\"Compile IPynb into HTML.\"\"\"\n",
"\n",
" name = \"ipynb\"\n",
" supports_onefile = False\n",
"\n",
" def compile_html(self, source, dest, is_two_file=True):\n",
" if flag is None:\n",
" req_missing(['ipython>=1.1.0'], 'build this site (compile ipynb)')\n",
" makedirs(os.path.dirname(dest))\n",
" HTMLExporter.default_template = 'basic'\n",
" c = Config(self.site.config['IPYNB_CONFIG'])\n",
" exportHtml = HTMLExporter(config=c)\n",
" with codecs.open(dest, \"w+\", \"utf8\") as out_file:\n",
" with codecs.open(source, \"r\", \"utf8\") as in_file:\n",
" nb = in_file.read()\n",
" nb_json = nbformat.reads_json(nb)\n",
" (body, resources) = exportHtml.from_notebook_node(nb_json)\n",
" out_file.write(body)\n",
"\n",
" def create_post(self, path, **kw):\n",
" # content and onefile are ignored by ipynb.\n",
" kw.pop('content', None)\n",
" onefile = kw.pop('onefile', False)\n",
" kw.pop('is_page', False)\n",
"\n",
" makedirs(os.path.dirname(path))\n",
" if onefile:\n",
" raise Exception('The one-file format is not supported by this compiler.')\n",
" with codecs.open(path, \"wb+\", \"utf8\") as fd:\n",
" fd.write(\"\"\"{\n",
" \"metadata\": {\n",
" \"name\": \"\"\n",
" },\n",
" \"nbformat\": 3,\n",
" \"nbformat_minor\": 0,\n",
" \"worksheets\": [\n",
" {\n",
" \"cells\": [\n",
" {\n",
" \"cell_type\": \"code\",\n",
" \"collapsed\": false,\n",
" \"input\": [],\n",
" \"language\": \"python\",\n",
" \"metadata\": {},\n",
" \"outputs\": []\n",
" }\n",
" ],\n",
" \"metadata\": {}\n",
" }\n",
" ]\n",
"}\"\"\")\n",
"```\n",
"The implementation use the power of the `IPython.nbconvert` library to render the `ipynb` file content in suitable HTML used later by **Nikola** itself to build the blog post/site.\n",
"\n",
"* **Nikola-IPython** console.\n",
"\n",
"**Nikola** also embed the IPython console in its system and pass some context to it.\n",
"\n",
"```python\n",
"def ipython(self, willful=True):\n",
" \"\"\"IPython shell.\"\"\"\n",
" try:\n",
" import IPython\n",
" except ImportError as e:\n",
" if willful:\n",
" req_missing(['IPython'], 'use the IPython console')\n",
" raise e # That\u2019s how _execute knows whether to try something else.\n",
" else:\n",
" site = self.context['site'] # NOQA\n",
" conf = self.context['conf'] # NOQA\n",
" commands = self.context['commands'] # NOQA\n",
" IPython.embed(header=self.header.format('IPython'))\n",
"```\n",
"\n",
"In this way you can quickly fire a **Nikola-IPython** console and use some of the Nikola features in a very simple way (probably more useful for more advances users).\n",
"\n",
"```bash\n",
"(stable)damian@damian-S400CA:/media/datos/Desarrollos/damian_blog$ nikola console\n",
"Scanning posts......done!\n",
"Python 2.7.4 (default, Apr 19 2013, 18:28:01) \n",
"Type \"copyright\", \"credits\" or \"license\" for more information.\n",
"\n",
"IPython 2.1.0 -- An enhanced Interactive Python.\n",
"? -> Introduction and overview of IPython's features.\n",
"%quickref -> Quick reference.\n",
"help -> Python's own help system.\n",
"object? -> Details about 'object', use 'object??' for extra details.\n",
"\n",
"\n",
"Nikola v7.0.0 -- IPython Console (conf = configuration file, site = site engine, commands = nikola commands)\n",
"\n",
"In [1]: conf.keys()\n",
"Out[1]: \n",
"[u'FEED_LENGTH',\n",
" ...\n",
" u'PRETTY_URLS']\n",
"\n",
"In [2]: site.THEMES\n",
"Out[2]: [u'zen-ipython', 'zen-jinja', 'base-jinja', 'base']\n",
"\n",
"In [3]: commands.\n",
"commands.auto commands.import_wordpress\n",
"commands.bootswatch_theme commands.install_theme\n",
"commands.build commands.list\n",
"commands.check commands.main\n",
"commands.clean commands.new_page\n",
"commands.console commands.new_post\n",
"commands.deploy commands.orphans\n",
"commands.doit_auto commands.plugin\n",
"commands.dumpdb commands.serve\n",
"commands.forget commands.strace\n",
"commands.github_deploy commands.tabcompletion\n",
"commands.help commands.version\n",
"commands.ignore \n",
"\n",
"In [3]: commands.new_post?\n",
"Type: new_post\n",
"String form: <nikola.utils.new_post object at 0x316f710>\n",
"File: /media/datos/virtualenvs/stable/local/lib/python2.7/site-packages/nikola/utils.py\n",
"Definition: commands.new_post(self, *args, **kwargs)\n",
"Docstring:\n",
"Function wrapper for command new_post\n",
"arguments:\n",
"is_page type bool default False\n",
"title type str default \n",
"tags type str default \n",
"onefile type bool default False\n",
"twofile type bool default False\n",
"content_format type str default \n",
"schedule type bool default False\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Bidirectional Integration\n",
"\n",
"* Creating a blog post with **Nikola** and **IPython**.\n",
"\n",
"The easiest way to do it is just calling the proper **Nikola** command using the `!` to pass it from the **Nikola-IPython** console to the shell.\n",
"\n",
" In [1]: !nikola new_post -f ipynb\n",
" \n",
"But we can do it better ;-) \n",
"We can pass arguments to the `nikola new_post` command such as the `title`, `tags`, `format` (we have already pass the format with `-f ipynb`), etc. So, let me create some variables for these arguments inside my **Nikola-IPython** console session:\n",
"\n",
" In [1]: title = \"Bidirectional IPython-Nikola workflow to write your blog post\"\n",
" \n",
" In [2]: tags_list = ['python', 'IPython', 'nikola', 'blog', 'extension', 'gh-pages', 'git', 'workflow']\n",
" \n",
" In [3]: tags = ', '.join(tags_list)\n",
" \n",
" In [4]: tags\n",
" Out[4]: u'python, IPython, nikola, blog, extension, gh-pages, git, workflow'\n",
"\n",
"Here we have the string `title` containing the title for the current blog post and a list called `tags_list` containing the corresponding tags. Later we call the `.join` method because we will need to pass a string with the tags separated by commas to the `nikola new_post` command.\n",
"\n",
"But, **wait a minute**... we have **Python** variables containing the title and the tags for my blog post. How can pass this variables to the shell to use them as arguments of the `nikola new_post` command???\n",
"\n",
"OK, to solve this problem we will use an exciting feature from **IPython**, just see the following line:\n",
"\n",
" In [5]: !nikola new_post -f ipynb -t \"{title}\" --tags=\"{tags}\"\n",
"\n",
"which will create the desired `ipynb` and the corresponding `.meta` file containing the title and tags.\n",
"\n",
" Creating New Post\n",
" -----------------\n",
" \n",
" Title: Bidirectional IPython-Nikola workflow to write your blog post\n",
" Scanning posts......done!\n",
" Your post's metadata is at: posts/bidirectional-ipython-nikola-workflow-to-write-your-blog-post.meta\n",
" [2013-10-15T19:21:28Z] NOTICE: new_post: Your post's text is at: posts/bidirectional-ipython-nikola-workflow-to-write-your-blog-post.ipynb\n",
"\n",
"*Que me contusi!* (or... do you see the bidirectional nature of this interaction? Nice, don't you think?)\n",
"\n",
"Yes, inside **IPython**, we can use the braces to pass **Python** variables to the shell... ;-) \n",
"\n",
"*Do you see the possibilities we have with this little feature?*\n",
"\n",
"Now, I *cd* into the post folder of my site and open the **IPython notebook** with my custom extensions.\n",
"\n",
" In [6]: cd posts/\n",
" /media/datos/Desarrollos/damian_blog/posts\n",
" \n",
" In [7]: !ipython notebook --profile=myext\n",
"\n",
"In the **IPython Dashboard**, we will open the new `ipynb` (inside the post folder) called `bidirectional-ipython-nikola-workflow-to-write-your-blog-post.ipynb` and write the content.\n",
"\n",
"After writing your content, **Nikola** has a `nikola deploy` command which you can customize with your conf.py file. You can see it here my configuration:\n",
"\n",
"```python\n",
"# Commands to execute to deploy. Can be anything, for example,\n",
"# you may use rsync:\n",
"# \"rsync -rav output/* [email protected]:/srv/www/site\"\n",
"# And then do a backup, or ping pingomatic.\n",
"# To do manual deployment, set it to []\n",
"DEPLOY_COMMANDS = [\"git add .\",\n",
" \"git commit -am 'Update'\",\n",
" \"git push origin master\",\n",
" \"git subtree split --prefix output -b gh-pages\",\n",
" \"git push -f origin gh-pages:gh-pages\",\n",
" \"git branch -D gh-pages\"]\n",
"```\n",
"\n",
"The instructions are very easy to understand, if you need more details just read this [post](http://www.damian.oquanta.info/posts/one-line-deployment-of-your-site-to-gh-pages.html) I wrote some months ago.\n",
"\n",
"We are essentially pushing the content to master, creating a split subtree and deploy it later to **gh-pages**.\n",
"\n",
"OK, but because we are *lazy* (I am ;-)), some months ago I wrote an **IPython** *javascript* nbextension which let me execute this `nikola deploy` command from a button in the **IPython notebook toolbar**. Essentially something like this:\n",
"\n",
"```javascript\n",
"function nikolaDeploy(path, clean) {\n",
" IPython.notebook.kernel.execute('cd ' + path);\n",
" if (clean==\"True\") {\n",
" IPython.notebook.kernel.execute('!nikola clean');\n",
" }else{\n",
" //do nothing\n",
" }\n",
" IPython.notebook.kernel.execute('!nikola build');\n",
" IPython.notebook.kernel.execute('!nikola deploy');\n",
" messager();\n",
"}\n",
"```\n",
"\n",
"![](https://gist.githubusercontent.com/damianavila/41030242043536bb25ce/raw/78c7c16adbbdef3389542add48d279ae2c38c578/nikipy_6.png)\n",
"\n",
"As you can see, we call `nikola clean` to clean my site, then we built it with `nikola build`, and deploy it with my customized `nikola deploy`... and all these actions with just a click!\n",
"\n",
"And now, you have the complete workflow! A very simple one, because you can make a lot of modifications to improve it. However, this workflow show you how you can integrate two exciting projects, communicate one with the other and vice versa, and get beautiful results."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# *NikIPy* Bridge\n",
"\n",
"But you can use this **Nikola-IPython** bidirectional workflow not only locally but also you can setup and environment in the cloud and write your blog post from everywhere in the world (and outside teh world too ;-), as long as you have connectivity).\n",
"\n",
"Recipe:\n",
"\n",
"1. Setup an **IPython** notebook server on the web (you can use **miniconda** or **Anaconda** solutions to easily install all the needed requirements or you can use **Wakari**, which is a useful, ready to go solution because you only have to open an account and the **IPython** notebook is inside the environment without any additional installation).\n",
"2. You will need git, and a **Github** account to get access to the **gh-pages** service, a simple solution to host your static site (obviously, you can use other services opr deploy the blog/site in your own server too).\n",
"3. You will need to install **Nikola** and some **IPython**-based theme, ie. my **zen-ipython** theme, but you have other ones. \n",
"4. Finally, setup an easy access to write you blog post...\n",
" a. Access to your **IPython** notebook server on the web (**IPython**).\n",
" b. Create an index ipynb to easily create new blog posts as I have shown before (**Nikola-IPython** workflow).\n",
" c. Navigate to your new generated post and write the content (**IPython** and **You!**. And some help from the **zen mode** and the **spellchecker** extensions, two **IPython** nbextension to make you life easier when you write your content).\n",
" d. Deploy the content to the **gh-pages** branch (**IPython** nbextension to deploy content with just a button, using the **Nikola** deploy command under the hood).\n",
" e. Enjoy your content on the web available fot the World!\n",
"\n",
"![](https://gist.githubusercontent.com/damianavila/41030242043536bb25ce/raw/8f2081ba396dd23f70998aa4bb7e1457b359f24a/nikipy_7.png)\n",
"\n",
"![](https://gist.githubusercontent.com/damianavila/41030242043536bb25ce/raw/c70c2050207c231e6d3a4d7a161058d4d2f5b830/nikipy_8.png)\n",
"\n",
"As you can see, all the pieces are there... but you are probably thinking. I want an automatic setup, I don't want to do all these things just ONCE. OK... this is what we are working on ;-). A **NikIPy** platform, easily installable and ready to go! Just be patient! You can play with all these ingredients in the meantime. You will have a lot of fun!!!\n",
"\n",
"Thanks.\n",
"\n",
"* **Dami\u00e1n Avila**\n",
"* **@damian_avila**\n",
"* **[email protected]**\n",
"\n",
"# Useful links\n",
"\n",
"\n",
"* http://getnikola.com/\n",
"\n",
"* http://ipython.org/\n",
"\n",
"* http://www.damian.oquanta.info/posts/bidirectional-ipython-nikola-workflow-to-write-your-blog-post.html\n",
"\n",
"* http://www.damian.oquanta.info/posts/deploy-your-nikola-powered-blog-content-from-the-ipython-notebook.html\n",
"\n",
"* http://www.damian.oquanta.info/posts/a-poor-man-spell-checker-for-the-ipython-notebook.html\n",
"\n",
"* http://www.damian.oquanta.info/posts/zen-mode-extension-for-the-ipython-notebook.html\n",
"\n",
"* http://www.damian.oquanta.info/posts/zen-themes-updated.html\n",
"\n",
"* http://www.damian.oquanta.info/posts/ipython-plugin-for-nikola-updated.html\n",
"\n",
"* http://www.damian.oquanta.info/posts/blogging-with-nikola-and-ipython.html"
]
}
],
"metadata": {}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment