sile-superdefine: README.md (draft)

sile-superdefine.md

A super \define for SILE

sile-superdefine levels up your \define command, making it more like LaTeX's!

It has these main benefits:

1. It allows you to redefine commands, and make \define's that refer to themselves.
2. It allows you to pass arguments from parents to children, via \args.
3. It allows you to call \process multiple times.
4. It allows you to feed it multiple \process blocks, see § \processn.
5. It allows you to run Lua patterns anywhere, but primarily on the define body. See § \gsub.
6. You can refer to individual arguments as e.g. \arg-color, and remove/add to \args array. See § \args.

You may also be interested in sile-texmode, if you think that SILE should be more TeX-y.

Example

\begin[papersize=50mm x 30mm]{document}
\script[src=packages/superdefine] % redefines \define
\script[src=packages/url]
\define[command=href]{\color[color=#000099]{\href[\args]{\process}}}
\define[command=repeat]{\process\process}
\neverindent
Check out my website: \href[src=http://tagalog.pw]{Tagalog.pw}!
\repeat{I love SILE! }
\end{document}

Nota bene

The SILE developers aren't at all amused by this package!

They recommend you write inline \script's, or include internal per-document packages. You might want to heed their advice.

API

\processn

sile-superdefine can be fed multiple \process blocks, like this:

% For sanity, `n` tells `\define` how many to expect.
\define[command=reverse, n=2]{\process2\process1}
\reverse{{A}{B}} % outputs "BA"

You can also tell sile-superdefine to output the rest of the blocks:

\define[command=omit3, n=-1]{\process4\process}
\define[command=omit2rev1, n=-1]{\process4\process3\process}
\define[command=omit2rev1rep, n=-1]{\process4\process3\process\process}
\define[command=rep, n=-1]{\process\process}
\define[command=noop,  n=-1]{\process}

\omit3{{1, }{2, }{3, }{4, }{5, }{6}} -- outputs "4, 5, 6"
\omit2rev1{{1, }{2, }{3, }{4, }{5, }{6}} -- outputs "4, 3, 4, 5, 6"
\omit2rev1rep{{1, }{2, }{3, }{4, }{5, }{6}} -- outputs "4, 3, 4, 5, 6, 4, 5, 6"
\noop{{1, }{2, }{3, }{4, }{5, }{6}} -- outputs "1, 2, 3, 4, 5, 6"
\rep{{A}{B}} -- outputs ABAB

\args

The most common use of this package is to pass arguments from parent to child, as in \href in the example.

You can also do this more complicated variation:

\define[command=href]{\color[color=\arg-color]{\href[\argsminus{color}]{\process}}}

\argsminus passes all \args except color.

You can default an arg, because why not!:

\define[command=href, arg-color=#000099]{\color[color=\arg-color]{\href[\argsminus{color}]{\process}}}

\gsub

\gsub runs a Lua pattern using string.gsub on the input string.

Consider:

\define[command=\texquotes]{\gsub[pattern=%'%', rep=”]{\gsub[pattern=%`%`, rep=“]{\process}}}

(You probably want sile-texmode instead, this example is contrived.)

While meant to be used in \define bodies, it can be used anywhere in the document if sile-superdefine is loaded.

\gmatch

\gmatch returns 1 if the input string matches the Lua pattern, else it outputs nothing.

\if

You may often want to do one thing if the input or an arg matches a pattern, else another thing.

There is no \else. You instead pass \if three \process blocks, and it outputs the second if the condition, the first, is true,† else it outputs the third.

\neverindent
\define[command=redblue]{\if{{\gmatch[pattern=^Q]{\process}}{\color[color=red]{\process}}{\color[color=blue]{\process}}}

\redblue{Quitter} -- red

\redblue{Winner} -- blue

† In superdefinitions, truthiness is defined as "outputs something".

\for

Not yet implemented; maybe some day!

(Responding here instead of in DMs so open source discussion is in public: somebody may want to disagree or see something neither of us do.)

There are some legitimately cool ideas in here, but there is also a lot of making simple things more complicated than they need to be. I haven't even seen the source code for what you are talking about here but from the documentation I can say this isn't something I'd like to maintain as part of the SILE core distribution. You're welcome to pursue this for your own usage and anybody else who wants it, but I don't want to make the TeXlixe input language any more complex than it has to be to be an effective markup language. Notable, **I don't think it should turn into a Turing complete programming language. It's supposed to be markup, not a programming language. This comes awful close to enough mechanics to pass a Turing test. While that's a feat in itself and I actually admire that, I'm not sold on that being something we want to maintain.

SILE is already complex enough and used enough in production (multiple publishing houses using it exclusively) that breaking things between versions is a big deal. Function deprecation, syntax changes, etc. are a pretty big deal. I'm still struggling with how to get rid of stdlib from the class system and replace it with Penlight because the transition won't be an easy one for end users that need to port code to make their classes work.

Given that we already provide full programmatic access to every aspect of the typesetter via Lua I don't think we want to maintain our own language on top of that.

Besides, most of what you are trying to do here can already be done in ways I believe are simpler.

Take your benefit 3 (shown in the first example). You don't need any of this to accomplish the same thing, the current \define already does that just fine:

\begin{document}
\define[command=foo]{\process\process}
\foo{bar}ian
\end{document}

barbarian

Simpler no?

In a similar vein it would be trivial to implement functions that mess with content order in Lua, and much more flexible:

SILE.registerCommand("reverse", function (_, content)
  for i = 1, #content do
    SILE.process(content[#content + 1 - i])
  end
end)

As a bonus it handles however much content you throw at it. If you need more specific things like sorting or omitting arguments, Lua provides documented stable interfaces to iterate table data that don't require building your own language. I think this goes for all of the \process n examples you showed, all of those could be done more effectively in Lua. More control, more test and debug functions, more speed, more everything.

An argument pass-through for macros might make sense, but I definitely don't want to get into syntax that puts a SILE command inside an options group bracket. I.e. not this: \href[\args]{\process.}. Whatever we do about arguments this isn't it. This way lies insanity. It's only a small step from there to this:

\newcommand*\UsePlaceHolder[1]{\@ifundefined{ph#1}{\DeclarePlaceHolder{#1}}{}\global\csname ph#1\endcsname\csname phset#1\endcsname\relax\def\@phtempa{\expandafter\the\csname phset#1\endcsname}\global\expandafter\def\csname ph\@phtempa#1used\endcsname{}\@ifundefined{ph@\@phtempa#1value}{}{\csname ph@\@phtempa #1value\endcsname}}

That's real live production code from a a project I still use LaTeX for because SILE can't really handle its requirements as well yet. But the mixed nature of TeX as both a content markup language and a programming language makes it HIDEOUS to extend or maintain. That code isn't an eyesore just because I'm an idiot, I got help from some of the top names in the TeX world to accomplish what I wast trying to do there.

As a post script to those arguments, let me throw out a couple litmus tests:

• Anything that makes syntax highlighters (e.g. vim-sile) for TeXlike input is probably a no-no for SILE core. Pass-through blocks that actually pass through to another interpreter are different. We already have one for Lua and I'm working on ones for Fluent and Lilypond. I'd be happy to consider a multitude of others with the requirement that they actually get first class support for the language the embed. The \script tag doesn't take a subset of Lua code re-implemented by us, it is a genuine eval by Lua on the actual input. The same will be true for Fluent & Lilypond and beyond. Anything that just mimics another language or changes the overall syntax of the TeXlike input format probably won't make it into core.

• Anything that makes document format conversion harder than it already is will need serious justification. I'm working on a Pandoc reader/writer for SILE (actually I already use the Writer in production, but it isn't clean enough to merge). Right now it's quite possible to achieve round-trip conversion to and from SILE for many data structures. Anything that moves SILE outside that realm of possibility and into where LaTeX is (in that only subsets of the markup are recognizable) is probably a no-no.

Again feel free to do whatever you like in your own packages, and we might consider some things for bundled packages if they are implemented simply and robustly, but super-define as documented is not suitable for what we want a markup language to be.

@alerque Thank you for the thoughtful long comment. I'm sorry to reduce it to this, and I do plan to eventually respond in more detail, but you've uncovered a problem with the SILE documentation.

Take your benefit 3 (shown in the first example). You don't need any of this to accomplish the same thing [...] \process\process

(above)

You can’t call \process more than once within the same macro.

SILE manual v. 0.10.9, p. 50

So, even if it technically works for now, (I didn't try, just took manual at its word,) there's at least the benefit I won't break it, and will consider it being broken a bug :-)

Sure, the docs are out of sync with reality on this point. I actually saw the change go by but it was a side effect of other fixes — the other fixes plainly being better and the previous limitation of only being able to call \process once being a side effect of a buggy implementation. See #1006.