Main inspiration comes from [here][1].
“”” Here is what a deep learning system stack would look like in nowdays.
- Build operator level graph description language: name whatever dl frameworks you care about, and [ONNX][2]
- Tensor primitive level graph description languages: [NNVM][3], [HLO/XLA][4], [NGraph][5]. It is close enough to the first one that you can also build graph optimization on first layer and bypass this layer.
- DSL for description and codegen: TVM, image processing languages like [halide][6], [darkroom][7].
- Hardcoded optimized kernel library: [nnpack][8], [cudnn][9], [libdnn][10]
- Device dependent library: [maxas][11](assembler for NVIDIA Maxwell architecture)