Scratchpad for a simpler markdown-inspired language for compilation to a subset of regex

Markdown Regex

"[name] is [age:int] years old." -> "(?<name>[^ ]) is [(?<age>\d+) years old \."

----------------------------------------

Examples

"search [service]/[region] [metric]"
"search (?<service>[^ ^/]+)/(?<region>[^ ]+) (?<metric>[^ ]+)"

"stock (?<name>[^ ]) (?<op>((<|>|=)=?)) (?<value>[^ ]+)"
"stock [name] [op](<,<=,>,>=,==) [value]"

----------------------------------------
Questions

Constraints?
    [label:constraint]

Options?
    [label](opt,opt,opt)

Escape?
    \[not a \] label

Constraint Options
    [label:constraint](opt,opt,opt)

Chain Constraints?
    [label:cons:cons]

Chain Constarints with Options?
    [label:cons:cons](opt,opt:opt,opt)

Back/Forward Refs?
    NO.

Greed?
    Always non-greedy

Alternative constraint/option formatting:

[label:constraint(opt,opt,opt)]

Multi-constraint:

[label:const(opt,opt,opt):const:const(2)]

What does Multiconstraint mean? Ambiguous mapping to regex?

Edge detection? How does name define here? Constraint resolution order?

[name][op:any(<,>)][value:number]

Prior art - angled brackets instead of square brackets? Most routing matching, bottle, flask

<name><op:any(\<,\>)><value:number>

Possibly configurable?

• Pro: small set of constrained characters; open/close match, open/close constraint args, comma, label/constraint separator (currently colon)
• Pro: easy to drop in for languages that heavily use default characters (<> math, html) ([] markdown) ({} json)
• Con: more config to check when reading code since brackets/delims are non-uniform
• Con: non-uniform style means more support and learning questions (what config are you using?) and support confusion (no, I don't need to escape brackets because I'm using <>)

pattern /<foo>/<bar>/
input /hello/world/path/to/blah
output {'foo': ?, 'bar': ?}

<[tag]>[value]</[:ref(tag)]>

 |
translates
 |
 V

^<(?P<tag>.*?)>(?P<value>.*?)</(?P=tag)>$

 |
test
 |
 V

<title>>words and <small>subtitle here</small></title>

"""
simplex.py

    <[tag]>[value]</[:ref(tag)]>

    <[tag]>       [value]    </[:ref(tag)]>
        |             |            |
        V             V            V
 ^<(?P<tag>.*?)>(?P<value>.*?)</(?P=tag)>$
"""
import re

ESCAPE = "\\"
REFERENCE = "(?P<{name}>{match})"
BACKREF_SYNTAX = re.compile("ref\((?P<name>.*?)\)")
BACKREF = "(?P={name})"
STATES = [
    "START",
    "CONSTANT",
    "MATCH",
    "ERROR"
]


def build_const(match):
    return re.escape(match)


def build_match(match):
    pieces = match.split(":")
    if len(pieces) == 1:
        return REFERENCE.format(name=pieces[0], match=".*?")
    elif len(pieces) == 2:
        m = BACKREF_SYNTAX.match(pieces[1])
        if not m:
            raise ValueError("Unknown constraint " + pieces[1])
        return BACKREF.format(**m.groupdict())
    else:
        raise ValueError("Unknown match " + match)


def compile(string):
    state = "START"
    parts = ["^"]
    context = ""
    error = None

    i = 0
    n = len(string)
    while i < n:
        c = string[i]

        if c == "[":
            if state == "START":
                state = "MATCH"
                i += 1
            elif state == "MATCH":
                state = "ERROR"
                error = "Cannot nest matches"
            elif state == "CONSTANT":
                if context[-1] == ESCAPE:
                    context += c
                    i += 1
                else:
                    parts.append(build_const(context))
                    context = ""
                    state = "MATCH"
                    i += 1
        elif c == "]":
            if state == "START":
                state = "ERROR"
                error = "Unexpected match close"
            elif state == "MATCH":
                if context[-1] == ESCAPE:
                    state = "ERROR"
                    error = "Cannot nest matches"
                else:
                    parts.append(build_match(context))
                    context = ""
                    state = "START"
                    i += 1
            elif state == "CONSTANT":
                if context[-1] == ESCAPE:
                    context += c
                    i += 1
                else:
                    state = "ERROR"
                    error = "Unexpected match close"
        else:
            if state == "START":
                state = "CONSTANT"
            context += c
            i += 1

        if state == "ERROR":
            break

    # Clean up last state
    if state == "START":
        pass
    elif state == "MATCH":
        state = "ERROR"
        error = "Missing expected match close"
    elif state == "CONSTANT":
        parts.append(build_const(context))
        context = ""
        state = "START"
    parts.append("$")

    # Raise on errors
    if state == "ERROR":
        raise ValueError(error)

    uncompiled = "".join(parts)
    return re.compile(uncompiled)

bottom router:

import re
import simplex
import functools


class Router(object):
    def __init__(self, bot):
        self.bot = bot
        self.routes = {}
        bot.on("PRIVMSG")(self.handle)

    def route(self, pattern, function):
        pattern = simplex.compile(pattern)
        self.routes[pattern] = function

    def handle(self, nick, target, message):
        for pattern, func in self.routes.items():
            match = pattern.match(message)
            if match:
                fields = match.groupdict()
                func(nick, target, fields)
            return

    def match(self, pattern):
        return functools.partial(self.route, pattern)