GeneralNotes.ipynb

{
 "cells": [
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "# EuroSciPy 2015 - General Notes"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "# Tutorials"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## bokeh tutorial"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "*Really simple* splattable list"
  },
  {
   "metadata": {
    "collapsed": true,
    "trusted": true
   },
   "cell_type": "code",
   "source": "# Allows you to change all attributes of a list at the same time... - simple reimplementation of __setattr__\nclass SplattableList(list):\n    def __setattr__(self, attr, value):\n        for x in self:\n            setattr(x, attr, value)\n    \n    # This gives you tab completion, if (AND ONLY IF) all elements are the same type\n    def __dir__(self):\n        if len(set(type(x) for x in self)) == 1:\n            return dir(self[0])\n        else:\n            return dir(self)",
   "execution_count": 2,
   "outputs": []
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Look into bokeh.charts once it has all been updated...in a few months/year?"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## skimage tutorial"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "`img_as_float` is a *very* useful function. Will convert an image of *any* type into floats between 0 and 1.\n\n**TODO:** Check exactly how this does the scaling, will it do crazy things for my satellite images? "
  },
  {
   "metadata": {
    "collapsed": true,
    "trusted": true
   },
   "cell_type": "code",
   "source": "# Set default colormap to grey and interpolation to nearest - useful for img processing scripts\nplt.rcParams['image.cmap'] = 'gray'\nplt.rcParams['image.interpolation'] = 'nearest'",
   "execution_count": null,
   "outputs": []
  },
  {
   "metadata": {
    "collapsed": true,
    "trusted": true
   },
   "cell_type": "code",
   "source": "# Can load multiple images into a collection\nic = io.imread_collection('../images/*.png')\n# Could potentially extend this to do really cool Landsat stuff...?",
   "execution_count": null,
   "outputs": []
  },
  {
   "metadata": {
    "collapsed": true,
    "trusted": true
   },
   "cell_type": "code",
   "source": "# What is this?\nnp.flatnonzero\n# Equivalent to\na.ravel().nonzero()[0]",
   "execution_count": null,
   "outputs": []
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "The main base of skimage is the `ndimage` obj from `scipy`"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "I should potentially move to adaptive thresholding...See http://scikit-image.org/docs/dev/auto_examples/plot_threshold_adaptive.html#example-plot-threshold-adaptive-py and the `threshold_adaptive` function"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Look into other denoising algorithms (**particularly the bilateral filter...)\n\n* The [bilateral filter](https://en.wikipedia.org/wiki/Bilateral_filter) uses similar ideas as for the median filter or the average filter: it averages a pixel with other pixels in a neighbourhood, but gives more weight to pixels for which the gray value is close to the one of the central pixel. The bilateral filter is very efficient at preserving edges.\n\n* The [total variation](https://en.wikipedia.org/wiki/Total_variation_denoising) filter results in images that are piecewise-constant. This filter optimizes a trade-off between the closeness to the original image, and the (L1) norm of the gradient, the latter part resulting in picewise-constant regions. "
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "**ACTION:** Generally, the second-half of euroscipy/skimage/skimage-tutorials-euroscipy2015/lectures/adv5_blob_segmentation.ipynb is particularly useful - READ IT MORE!"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "**ACTION:** Try adding some of these denoising methods to my PhD process...and try looking into how we can do adaptive thresholding, or adaptive segmentation, or adaptive edge detection. Smoothing or filtering is 'bad' in some ways as we don't want to actually change the values (as these are used for our AOT estimation), so maybe we do segmentation on a smoothed or cleaned image, and then overlay that segmentation on the original image.\n\nBear in mind that we are already doing a 3x3 smoothing (3x3 median filter) on the resulting AOT image...so maybe it won't matter if we do some denoising earlier? However, then we are removing info anyway...it could be genuine high AOT..."
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "#### Processing batches of images\n\nIf one wishes to process a single image, a lot of trial and error is possible, using interactive sessions and intermediate visualizations. Such workflow typically allows to optimize over parameter values, such as the size of the filtering window for denoising the image, or the area of small spurious objects to be removed.\n\nIn a time of cheap CCD sensors, it is also frequent to deal with collections of images, for which one cannot afford to process each image individually. In such a case, the workflow has to be adapted.\n\n* function parameters need to be set in a more robust manner, using statistical information like the typical noise of the image, or the typical size of objects in the image. ('semi-automatic data-mining of params')\n\n* it is a good practice to divide the different array manipulations into several functions. Outside an Ipython notebook, such functions would typically be found in a dedicated module, that could be imported from a script.\n\n* applying the same operations to a collection of (independent) images is a typical example of embarassingly parallel workflow, that calls for multiprocessing computation. The [``joblib``](https://pythonhosted.org/joblib/) module provides a simple helper function for using multiprocessing on embarassingly parallel ``for`` loops. \n\nLet us first define two functions with a more robust handling of parameters."
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "**ACTION:** Add joblib caching to PhD code"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Investigate `xray` and `yt` (sp?)"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "**ACTION:** Look into how `apply_parallel` (see http://scikit-image.org/docs/dev/api/skimage.util.html#apply-parallel) works. It seems to try and deal with boundary issues...will this be ok for some of my segmentation/correction stuff? (Maybe not seg, but correction?)"
  },
  {
   "metadata": {
    "collapsed": true
   },
   "cell_type": "markdown",
   "source": "## sklearn tutorial"
  },
  {
   "metadata": {
    "collapsed": true,
    "trusted": true
   },
   "cell_type": "code",
   "source": "# Like value_counts...gets freq of unique values\nyFreq = scipy.stats.itemfreq(y)",
   "execution_count": null,
   "outputs": []
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## cython tutorial"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Best idea is to test in IPython using `%%cython`. The `-a` argument to the cell magic will automatically show the highlighted HTML below the cell, where 'more yellow' = 'more Python, less C'. Try and remove as much yellow as possible (eg. remember to declare local variables, etc)"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## teaching\nBy *Greg Wilson*\n\n"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Two biggest demotivators for learning: indifference (teacher really doesn't seem to care) and unfairness (even if you get the preferential treatment).\n\nNo evidence for learning styles at all! Person who first suggested it has now retracted it...\n\n*How learning works* is a great book (by Ambrose). *Building a better teacher* (by Elizabeth Green) is also good.\n\nIn the US the avg school teacher has another teacher in their classroom 4-5hrs per year, and just for evaluation/promotion purposes. In Finland the avg school teacher has another teacher in his classroom 3-4hrs per week.\n\n*Teaching what you don't know* (by Huston) - great book. Includes funny stories, and you'll see yourself in the book!\n\nSee http://cacm.acm.org/blogs/blog-cacm/189498-top-10-myths-about-teaching-computer-science/fulltext too\n\nAlso see a good paper on this: http://www.pnas.org/content/111/23/8410.full.pdf\n\nAnd another paper with practical examples: http://files.software-carpentry.org/training-course/2013/08/p34-porter.pdf. For example:\n\n * Use pair programming - it improves outcomes, *and* is also used in the real-world\n * Media programming - don't get new programmers to write a loop to print times tables... Instead, use a few functions that open images, resize them and save them. They actually have practical experience in what you would expect to see. Similarly, in Software Carpentry, first Python stuff is to load in a CSV file, plot it, get max & min, and then do that in a loop for a load of files. Then we can go into merging files, checking shapes of files, etc. 'Transference' is the process of transferring this knowledge to other contexts, and yes this does transfer.\n * Peer instruction - thinking about an answer, then trying to convince someone else...this is where the learning happens. Learning by asking questions too. If people mis-understand things then it is likely that many people misunderstand in the same way... We don't really do this in Software Carpentry - mainly because it's v different to what the instructors are used to, and it's not what the learners are used to! Introducing something completely new as part of the introduction of a new topic (like programming) confuses things...eg don't use the IPython notebook as part of intro to progamming.\n * Combining all of the above - they are additive in terms of effect!\n \nUnfortunately, most people would rather fail than change...\n\n**Not** moving to a different room at the end of each lecture makes learning outcomes go down...it gets oxygen to the brain etc.\n\nAs a rough approx, human memory has two levels: long-term and short-term. Long-term memory is basically unbounded, and you can keep filling it up until you die. It works via pattern-matching and is slow (hundreds of milliseconds) - not fast enough to keep you alive. Short-term memory is a lot smaller, works via pattern-matching but can only hold a few things. Basically 7 +/- 2 discrete things, for about 10-12 seconds - and that's it! This explains the length of phone numbers all across the world!\n\nTeams larger than around 6-7 people then they tend to split into two or more teams. Sixes and Sixers in scouts etc!\n\nThrowing 50 ideas at a lecture room in 50 minutes is pointless... You need to introduce a few inter-related concepts and reinforce them.\n\nInner loop when teaching: Introduce half a dozen new things, densely connected, and can be kept as a 'chunk'. Repeat a few times, then stop at 45mins to have a break etc. Then repeat for more lectures.\n\nYou don't want to have silence in the room while you're doing something...eg when you're clearing the whiteboard get them to do something."
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Things you cover when teaching a for loop in python:\n\n * Concept of repeating things\n * Lists/collections of items\n * 'Stepping through'/'Incrementing' through the collection\n * The difference between the whole list and the particular item you're working on\n \nNotes:\n\n * Using sensible terms - not `for i in list` or `for i in range(5)`, but `for student in course` or `for molecule in material`.\n \n * The idea of 'visiting' each item in the list and doing something to it\n \n * In some ways C-style loops are easier?\n \n * The *action* or *body* of the loop is key, and often that is made trivial (`print`) etc\n \n * Can consider three components: loop var, collection, and body. Loop var takes values from collection, body does something with the loop var. \n \n * Need to include rationale... For each image in the directory, create a thumbnail. For each data file, do your processing etc.\n \nWe've got around 5-6 things here...this is a concept map. Blobs are the ideas, arrows are the relationships (with labels explaining how they're related). Need to draw these when you're prepping the lesson. **DO NOT:** Do more than 7 bits (blobs + arrows) in one chunk before reinforcement.\n \nYou can deal with the concept map (which is a Directed Graph) in many orders...and it's better to make notes like this, as when you do bulleted lists you tend to keep in the same order. The connections *must* be explicit, and must be *first class citizens*...they are what sticks and helps everything else stick.\n\nDo an exercise when you get back to your research group: explain concept maps, do a very simple concept map of the project we're working on. Do this and you'll find everyone's maps look very different... This 'externalises' the cognition - so you can share your thoughts easily and argue about it by pointing at different bits. Then you get a common understanding!\n\nCan also draw the concept map while you're doing the teaching...drawing it as you go. But you **definitely** should draw out the concept map before you do any teaching.\n\nMost of us teach at the speed we talk, rather than the speed that people actually learn."
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Three categories of people:\n\n * **Novice** - they don't know what they don't know ('unknown unknowns'). Lack a functional mental model of the problem domain - no map of the terrain. Their questions aren't even wrong, they're just completely un-understandable. They've cobbled together bits of various other mental models to try and get something working in their brain. The aim is *not* to give them lots of information - they will try and fit this into their existing mental models. 'Putting weight into the mental model boxes so they're harder to move around'. \"Never hesitate to sacrifice truth for simplicity\"...for a while (until they get to competent).\n \n * **Competent practitioner** - \n \n * **Experts** - Many experts actually know less facts than competent people. They key distinction between experts and competent people is that the experts 'knowledge graph' is more densely connected...many more linkages between the facts. Direct connections that don't go through lots of intermediate steps... As an expert you *don't know* how you got from A->G, and so you can't explain to others how you got there. You come up with some weird false path between A->B->C->D etc - which may not make any sense to the learner. This is 'expert blindness'. To get to expert-level you have to do 'reflective practice' - there has to be a feedback loop! Feedback needs to be real-time or near-real-time, so that it can be used to improve things. 'Competent people' are normally far better teachers than experts.\n \nSoftware Carpentry tries to get people from novice to competent, and doesn't think about going from there to expert. Impossible to develop expertise 'at scale'."
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "So many people *ONLY* believe in summative assessment, not formative assessment. They have to try and get the idea that \"I'm trying to help you\" rather than \"I'm judging you\".\n\nMCQs can be used for formative assessment... They're really fast. The answers need to be chosen carefully: one and only one right answer, all of the wrong answers ('distractors') need to be plausible so they look like the right answer. The wrong answers should have *diagnostic power*...they need to tell you *what the learner has misunderstood* - and where their mental model is wrong. Writing good answers requires you to *think* about what people might get wrong - a really useful thing to do!\n\n**Reverse instructional design** When you design a course, figure out what the final assessment is going to be. You need to establish how you're going to decide whether the learning has taken place before you can actually start teaching! Basically TDD but for teaching...\n\n**DO NOT** use `range` when introducing Python!\n\nSimilarly, only placing a single `print` statement in all of your example loops gives people the idea that you can 1) only print things out in a loop and 2) only have a single statement inside a loop (particularly in Python, as it is just indented).\n\nKinda like defensive programming - anticipate the errors and put things in place to stop them occurring."
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "MCQ to see whether the for loop teaching has worked - with diagnostic power of distractors.\n\nGiven this code:\n\n```ages = [14, 26, 37, 76]\nfor current_age in ages:\n    print(\"Current age is \", current_age)\n```\n\nDoes it print:\n\n**A:**\n\nCurrent age is 76\n\n**B:**\n\nCurrent age is 14, 26, 37, 76\n\n**C:**\n\nCurrent age is 14\n\nCurrent age is 26\n\nCurrent age is 37\n\nCurrent age is 76\n\n**D:**\n\nCurrent age is 37\n\nCurrent age is 14\n\nCurrent age is 26\n\nCurrent age is 76"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Most useful thing to ask other teachers before teaching something is \"What have you seen go wrong?\". Really important to get a collection of these stored somewhere..."
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "Send your MCQ to Gael!"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "\"You two sit next to each other and pair program, but only Person A is allowed to touch the keyboard\" - particularly where Person A is the lesser able."
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "# Conference"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Keynote 1\nData Cleaning on text to prepare for analysis and machine learning"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "*ZeroMQ guide* - O'Reilly, Read Ch 6 of this book, because it explains all of this community building. See http://zguide.zeromq.org/page:all#toc130\n\nGoal is to get people to work together, rather than to write lines of code: this changes your perspective on what you need to do.\n\nThe wrong question when someone contributes is \"Can I review your PR?\", \"What are your credentials?\". The right response is \"Wonderful - thank you for getting involved. How can we help you (to help us)?\". Comments about style guides and indentation aren't helpful!\n\nMaybe **just merge** PRs without worrying about whether they are good/bad etc. Just merge them, and then fix any problems with later patches. People worry about vandals and rubbish - but people worried about that in Wikipedia and wikipedia works!\n\nThis doesn't mean don't do code review...just separate the two stages. Merge first, then do code review.\n\nGetting a PR merged makes users happy.\n\n**Q:** But doesn't this mean that you end up with a very messy history...\n\n**A:** \"I don't clean history, I record history. Stalin 'cleans history'.\"\n\n"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Data Cleaning on text to prepare for analysis and machine learning\nIan Ozsvald,https://speakerdeck.com/ianozsvald/data-cleaning-on-text-to-prepare-for-analysis-and-machine-learning-at-euroscipy-2015\n\nMost industrial problems are messy, not hard - they're just time-consuming and annoying!\n\ntextract - a Python tool to pull text out of various different formats\nCERMINE - academic paper parsing - see http://cermine.ceon.pl/about.html\n\nmessytables (Python) - autoguesses dtypes for CSV, HTML, PDF tables etc\n\nFixing badly encoded text is a REAL PAIN IN THE ARSE!\n\nThere are 20-30 different unicode symbols that all look very much like a hyphen/dash!\nSomething like ftfy can help: https://github.com/LuminosoInsight/python-ftfy\n`unidecode` gets rid of some of this stuff.\n\n*Could you write a module to suggest possible conversions on a dataframe for the user - and notify if there are problems with MM/DD and DD/MM* - this just goes wrong silently. Could go even further: looking for categories automatically - and switching to categorical types. Could automatically create boolean equivalents.\n\nAutomated validation of data based on what we've seen before? **THIS COULD BE USED FOR FLOWMINDER** Eg. if we get new data for 2015, is it roughly as valid as the 2014 data? Eg. has it got the same perc of missing data, have data types changed, have we got weird data...\n\nOn text data pd.merge often doesn't work. Things like fuzzywuzzy/metaphone do things on phonetic spelling or character distances **COULD WORK REALLY WELL FOR ADMIN NAMES IN NEPAL**.\n\ndedupe.readthedocs.org could be really useful\n\nLOTS OF UNIT TESTS is important\n\nTool called `annotate.io` (will be OSS by the end of the day!). Automatically finds valid combination of operators to go from what you start with (\"53K with benefits\") to what you want (\"53000\").\n\n`glueviz` has linked cells for highlighting. Nice data investigation tool.\n\nsetosa.io/csv-fingerprint\n\nCould similar techniques be applied to non-text data? Eg. checking satellite data for insanities? checking other numeric data? spectra? all sorts of things?\n\nengarde is awesome - puts restrictions on your data frame!"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Big data in little laptop: a streaming story in Python (featuring toolz)\nJuan Nunez-Iglesias\n\n*Medium data* - fits on your hard disk but not in your RAM.\n\nSimple streaming of data:\n   \n```\nfor line in file:\n    dosomething()\n```\n\nCan stream using `yield` too. Nesting functions using yield means that the whole chain of functions is done once per line.\n\n`toolz` makes streaming beautiful\n\nCan just use\n\n    import toolz as tz\n    \n    tz.pipe(f1, f2, f3 etc)\n    \nCan use `cmap` - which deals with giving functions less arguments than it is expecting.\n\n"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Massively parallel implementation in Python of a pseudo-spectral DNS code for turbulent flows\nMikael Mortensen\n\nMain Q: is it possible to write a competitive, state of the art turbulence solver from scratch in Python? (NOT just wrapping a FORTRAN version) - using numpy, scipy, mpi4py etc.\n\n100 lines of Python can get you there...!\n\nUse mpi4py, with numpy arrays - at the speed of C!\n\n3D FFT in MPI only requires one communication - All to All (obv after initial allocation)\n\nufuncs can be slow...is this where `numba` comes in?"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## klepto: unified persistent storage to memory, disk, or database\nMike McKerns\n\nBasically abstract your storage whether it is on database, disk etc - and let it work nicely in parallel and over networks etc\n\n`dict`-style interface - can store functions, lambdas etc in any of these formats.\n\nVariety of backends, with same interface: dict, directory, file, sql etc.\n"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## pyMIC: A Python Offload Module for Intel(R) Xeon Phi(tm) Coprocessors\nMichael Klemm\n\nPython & HPC - HPC is too conservative! But still Python is picking up in terms of use...\n\nIntel Xeon Phi is a separate numeric co-processor - we have some in the Iridis cluster\n\nPyMIC runs *native compiled kernels* on the Phi.\n\nBuilt on top of raw C/C++ libraries, through Cython, and then low-level data management library in Python (`memcpy` style), then a nice high-level interface. Also a handy set of built-in kernels...\n\nBasically just:\n\n  1. Get a device\n  2. Get a stream\n  3. Load a native kernel\n  4. Run the kernel on the device, using numpy arrays etc (will deal with all copying etc itself)\n  \nCan do the copying manually, using `stream.bind` etc.\n\nThe OffloadArray interface wasn't really mentioned much, but looks like it could be particularly useful: https://github.com/01org/pyMIC/blob/master/pymic/offload_array.py"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Bokeh for Data Storytelling\nBryan Van de Ven\n\nBindings to Bokeh from a number of languages - including `RBokeh`\n\nCan produce graphs from DOT files using bokeh...**look into this more!**"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Getting more out of Matplotlib with GR\nJosef Heinen, http://pgi-jcns.fz-juelich.de/pub/doc/EuroSciPy_2015/\n\nWrote a new MPL backend based upon GR. Integrated with IPython, can display continuous data streams, create animations on the fly, integrate with GUI, and simulatenous output to multiple output devices.\n\n    from os import environ\n    environ['MPLBACKEND'] = 'module://gr.matplotlib.backend_gr'\n\n    import gr\n    gr.inline('mov')"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Flexx: A pure Python GUI tookit based on web technology\nAlmar Klein\n\nSharing a GUI app is a bit of a pain...using `cx_freeze` and email to them. Sharing 'with the world' is harder - really need to go to a web app, but that requires reimplementing app in JS/HTML.\n\nWould be great if you could create an app once in Python, and then deploy it as a desktop app or a webapp. Can run webapp from a server or export it to a static HTML file. They can all be used in the IPython notebook too - so can interact with it from other IPython cells.\n\nApp is just a compound widget.\n\nUses `flexx.pyscript` which is a Python to JS converter, using the `py2js` function. Basically works as a way to write JS via Python.\n\n`flexx.react` is a reactive programming way of linking with signals...like Qt slots. Even more - as you can connect with glob-style strings - ie. connect to `blah.*.name`.\n\n**Q:** Can you compile to an executable?"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Spimagine – interactive GPU based visualization and processing of time lapsed volumetric data\nMartin Weigert\n\nDoes convolution on GPUs...**LOOK AT THIS:** https://github.com/maweigert/gputools/ which I think includes the convolution stuff that I need...\n\n`ndimage` significantly faster than `scipy.signal` for convolution! **ACTION:** Use this in HOTBAR! (See http://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.filters.convolve.html#scipy.ndimage.filters.convolve compared to http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.convolve2d.html)\n\nAlso see https://github.com/maweigert/spimagine for visualisation."
  },
  {
   "metadata": {
    "collapsed": true,
    "trusted": true
   },
   "cell_type": "markdown",
   "source": "## Keynote\nRandy Leveque\n\nQuiz-style things in notebooks - a function that takes an answer, checks it, gives you feedback, explains what went wrong etc.\n\nSageMathCloud: COMPLETELY FREE, even for teaching, at https://cloud.sagemath.com/\n\nnbstripout is written by minrk - handy git commit hook to strip out the output etc from notebooks for easier source control: https://gist.github.com/minrk/6176788\n"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Building complex visualizations easily for reproducible science\nPhilipp Rudigers\n\n*Stop plotting your data, let it visualise itself*\n\nCan do + or * to combine and/or overlay plots - handy!\n\nObjects like hv.HLine, hv.Image\n\nHoloMap for a range of params, can do interactive exploration, or use something like a hv.GridSpace to show them all on a grid.\n\nCan change backend to Bokeh *really* easily!\n\nVery extensible and composible - build everything from standard elements!\n\n**TODO:** Investigate this more! See http://holoviews.org/"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Dashboarding with the IPython notebook for online introspection into long-running experiments\nThomas Greg Corcoran\n\nDealing with long-running cluster jobs can be difficult...you have to deal with potential memory & disk issues, shared resource problems, bugs in your code crashing after 3 days, infrastructure is generally a pain!"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Probabilistic Programming and Sports Analytics\nPeadar Coyle\n\n`pymc3` is the new port of `pymc` - see https://github.com/pymc-devs/pymc3\nThis is in beta - based upon theano - a complete rewrite of PyMC2\n\nCurrently project to port Bayesian Methods for Hackers from PyMC2 to PyMC3 - and see Peadar's Github for a tutorial using PyMC3\n\nSee theano too - could be useful for various things: http://deeplearning.net/software/theano/\n\n"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "# Lightning talks\n\nEuroSciPy 2016 - Erlangen, nr Munich - end of Aug, last weekend.\n\nsphinx-gallery - see https://github.com/sphinx-gallery/sphinx-gallery\n\npandas: look at `.dt` accessor from a dt column, also look at `sample`, `pipe` and `assign`. `sort_values` is cleaning up the API.\n\nWomenHackNonProfits - @WHFNP (also linked to Women Who Code, Pyladies, Women Techmakers, STEMettes)\n\nhttp://pyladieslondon.github.io/\n\n*Elegant SciPy* book - talk to Juan and see https://github.com/HarrietInc/elegant-scipy-submissions\n\nSee Biggus: https://github.com/SciTools/biggus\n\nLook into AstroPy and associated libs - glueviz, astroML, astroquery, astroplan, photutils etc\n\nastropy.units is v useful, astropy.modeling has improvements over scipy (**incl for convolution**)\n\npdebuy/use_ipynb_git - can be a git filter to go and strip the output & cell numbers from the IPyNB **FLOWMINDER**\n\n"
  }
 ],
 "metadata": {
  "gist_id": "c6eb9739bfebf5430611",
  "language_info": {
   "mimetype": "text/x-python",
   "name": "python",
   "version": "2.7.9",
   "pygments_lexer": "ipython2",
   "file_extension": ".py",
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "nbconvert_exporter": "python"
  },
  "kernelspec": {
   "name": "python2",
   "display_name": "Python 2",
   "language": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}