Skip to content

Instantly share code, notes, and snippets.

@foxish

foxish/Looking Beyond Exaflops & Zettabytes.md

Last active November 30, 2015 23:16
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save foxish/475d89a8f1024cec05d9 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save foxish/475d89a8f1024cec05d9 to your computer and use it in GitHub Desktop.
Download ZIP
Looking Beyond Exaflops & Zettabytes by Dr. Pradeep Dubey, Director of the Parallel Computing Lab at Intel Corporation
Raw
Looking Beyond Exaflops & Zettabytes.md

Dr. Pradeep Dubey.

  • Synopsis:

    • Starting with hardware, went up till the application layer, with IBM moving "upwards" and become a services company.
    • Parallel computing lab, works with the end-user supercomputing applications, and not bottom-up.
    • Multicore -> Manycore -> Bigdata -> ML.
    • Why do we need to look beyond EZ (exaflops and zettabytes)?

  • Why machine learning?

  • ML drives innovation now.

  • How does it matter to people at large?

  • How do we make computing indispensible?

  • How good is a model?

    • We are modelling things in a way that our conceptual model is represented and then we can ask questions which is either mining (Is it..?) or synthesis (What if...)?
    • If the result is not good enough, can we iterate and get better over time?
    • Computing power is at the stage at which these models can be trained and incrementally updated is here.
    • When we need to do both forward and backward path, compute is the limiting factor.

  • BigData

    • The irregular data access pattern makes it a difficult problem.
    • There's also an HPC element, with large matrix computations.
    • Mining-learning is valuable, and that's what drives free storage, or promotes user upload of data, because they can reason about users better with more data.

  • Statistics

  • Annual predicted internet data rate = 1.6ZB

  • Total datacenter flops = ~1 Exaflop.

  • The compute doubles every 1.5 years and data every year.

  • Need for compute can only grow even further. The gap between data and compute is going to grow.

  • 1 PB of data takes 11 seconds to generate and 13 years to view as HD video!

  • Both data generation and consumption will be machine-led.

  • von Neumann

  • No low-hanging fruit anymore. It needs radical rethinking at the systems and the science levels.

  • Neuroscience and learning

  • Trying to understand how the brain forms synapses as it learns.

  • Reverse engineer the attentional states of the brain.

  • Creating real-time instrumentation to visualize human thought.

  • Requirements of an AI system

  • Supervised learning is what we see, but it's not scalable.

  • Strong unsupervised learning is needed, such that time to train scales linearly with input.

  • ML algorithms

  • K-means, naive-bayes, etc only coded at a high level, using toolkits where optimizations at a system level haven't even been considered.

  • The HPC perspective will help improve performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment