Npcomp notes - AST Extraction - June 6, 2020.md

Today I implemented enough boilerplate to parse and import a (very) simple python function to a corresponding MLIR representation.

As an example:

from npcomp.compiler.frontend import *

def binary_expression():
  a = 1
  b = 100
  c = a * b + 4
  return c

fe = ImportFrontend()
try:
  f = fe.import_global_function(binary_expression)
finally:
  print(fe.ir_module.to_asm(debug_info=True))

Prints:

module {
  func @binary_expression() -> !basicpy.UnknownType {
    %c1_i64 = constant 1 : i64 loc("python/samples/ast_extraction.py":4:6)
    %c100_i64 = constant 100 : i64 loc("python/samples/ast_extraction.py":5:6)
    %0 = "basicpy.binary_expr"(%c1_i64, %c100_i64) {operation = "Mult"} : (i64, i64) -> !basicpy.UnknownType loc("python/samples/ast_extraction.py":6:6)
    %c4_i64 = constant 4 : i64 loc("python/samples/ast_extraction.py":6:14)
    %1 = "basicpy.binary_expr"(%0, %c4_i64) {operation = "Add"} : (!basicpy.UnknownType, i64) -> !basicpy.UnknownType loc("python/samples/ast_extraction.py":6:6)
    return %1 : !basicpy.UnknownType loc("python/samples/ast_extraction.py":7:9)
  } loc("python/samples/ast_extraction.py":3:0)

While fairly basic, this illustrates a few points:

• Doing this kind of bridging is not terribly complicated, weighing in at ~250 lines of Python code.
• My priority is to keep the python side really thin -- preferring local AST mappings to MLIR wherever possible.
• We can then implement most of the legalization on the MLIR side, choosing the precise restricted python semantics we want to support.
• The location information is preserved and the MLIR diagnostics system is used for reporting errors (even at import time), which should give us some nice usability.
• While you could build a full python compiler with an approach like this, I'm actually interested in extracting program fragments from a running python session, which will require some more introspection of the python environment (and some more opinions to be taken down the line that wouldn't necessarily apply to compiling completely standalone modules/functions).