Notebooks are great for interactive use, not so much for being reused in an automatic fashion. How to make a notebook reusable (for example in an analysis pipeline)?
Write your notebook 'mynotebook.ipynb'
as usual. Then add a special comment in some cells:
### input
x = 0
y = 0
The cells with this comment work normally in the notebook. They let you initialize variables as you would normally do in a notebook. But in addition to that, these cells also serve to specify some variables as arguments to a reusable function, with default values.
...
At the end of the notebook, add another special comment to one or several cells.
### output
z
Typically, this is where you display the end result.
That's it! You've given all the necessary information to convert your interactive notebook into a reusable Python function. You can use it in a Python script (for example, a pipeline script) as this:
import nbfun
f = nbfun.load('mynotebook.ipynb')
z = f(x=1, y=2)
This nbfun
module doesn't exist yet but it should be easy to do.
The runipy module implements a similar idea.
.py files are definitely better for reusability. But you typically create your functions incrementally with a lot of trials and errors, and the notebook is great for that. The proposed solution just saves the burden of porting manually the code into a .py file and encapsulating the code into a function. This manual process would only take a couple of minutes, but if you integrate your notebook in a pipeline, you may need to repeat this dozens of times, as soon as you need to tweak something in your notebook.