slangs

gistfile1.txt

=head1 slangs as knitted DSL, doubling on sigils. 

The Perl language has the reputation to be line noise. 
This reputation is partly due to sigils. It is not deserved.
A sigil is the  character in front of an identifier.
Below, a Perl 5 oneliner that prints I<Hi Larry Wall>.
the identifier is C<$him> and the sigil is C<$>.

   my  $him='Larry Wall'; say "Hi $him"

The sigil allows the seamless interpolation of the
value hold by C<$him> in a doubly quoted string.
In another language, we need extra syntax to interpolate an expression.

   let  him='Larry Wall'; say "Hi {{him}}"

Also in shell, dealing with array gets hairy fast.
Not in Perl.

   my  @them=('Larry Wall', 'Jonathan Worthington';
   say "Hi $them[1]"


In Raku, double quoted strings really are code in a sublanguage, the P6Regex
slang, within the main language. The interpolated variables are really things of
the main language thightly knitted in the slang.

In Raku, we got twigils, or secondary sigils that denotes
the variable scope


Finally, we will talk of two slangs, one to generate an AST (abstract syntax
tree) and one to match within an AST. An AST is generated when compiling a
program before emitting bytecode or object code. At this point, we don't expect
you to understand much of the code below. The salient fact: we use 4 variables
in our code. C<$$zero>, the one with the double sigil, is special. It refers to the compiled
variable C<$zero>), not a variable in our compiler code. 


  my $ast0 := AST 0;
  my $ast := AST bind $$zero, $ast0;
  my $int-to-match = 0;
  if (ATM $ast ~~ bind $$zero, +$int-to-match   ) { ... };

Sigils, possibly doubled are one of the syntactical devices to
tigthly knit our slang together.


Perl up to Perl 5 may have its problems but they are not due to sigils
When a language is well designed, a rich syntax it more expressive.
The trade off is between expressivity and ease of learning.
Expressivity make program more concise, and most of all, more
readable to the savvy programmer.

Perl has borrowed sigils from Unix shells.
Raku regularize their usage and add twigils.
I use them to create slangs for compilers.
Slangs are the Raku take on Domain Specific Languages.
Slang mesh seamlessly within the main languages and within each other
and sigils are instrumental to that

The name sigil is proper to Perl but the concept predates
Perl. Sigils are already present in Unix shells and
prefix identifiers. Except they don't in the left hand side
of an affectation. Also the shell is fussy about spaces around the '=' sign. 


    $ him='Larry Wall'; echo "Hi $him" 
    Hi Larry Wall

    $ him='Larry Wall'; echo "Hi $him" 
    Hi Larry Wall


    perl -de 0
    $a=toto; print "hello $a"



A rakudo AST is gnerated by creating a tree of objets of C<QAST::*> classes.

   QAST::Op.new(:op<say>, QAST::IVal.new(:value(42));

   QAST::Op.new(:op<call>, :name<say>, QAST::IVal.new(:value(42)));

=Note
this is really a call in NQP, Not Quite Perl, a language used to bootstrap Raku.
An AST for a call in Raku would be more complex.


atm and ast are two slangs. ast, like its name suggest is used
to genrate ASTs. atm is used in optimizer to patterm match ASTs
so as to optimisiw them by rewriting them with ast.
Code of these slangs are respecitively introduced the phasers
AST and ATM.
If you know Perl or awk, you know the phasers BEGIN and END.
But code in AST and ATM are not in blocks. Or if they are, they
have a specific meaning

Say we want to write the code that generates the AST for the integer 0
and then matches that one node tree.

With our slangs, that's easy.
C<$ast> value is the one node tree and the atm expression
C<$ast ~~ 0> means we want the C<$ast> to match the C<IVal> for 0.

   my $ast := AST 0;
   if (ATM $ast ~~ 0 ) { ... };

Writing directly with the QAST API would be a mouthful.

   my $ast = QAST::IVal.new(:value(0));
   if nqp::istype($ast, QAST::IVal) && nqp::iseq_i($ast.value, 0);



   $_ = AST 0;
   if (ATM 0) { ... };


    



Below, A (so slightly) more realistic example.
We want to generate the AST code for C<$zero := 0>
but to match it with a twist. We want to match the integer
by passing it as the value of C<int-matched>. Also we
are building our AST piecemeal.
The two first lines are equivalent to C<my $ast = AST bind $$zero, 0>.
   
  my $ast0 := AST 0;
  my $ast := AST bind $$zero, $ast0;
  my $int-matched = 0;
  if (ATM $ast ~~ bind $$zero, +$int-matched   ) { ... };



   my $int-matched = 0;
   my $ast := QAST::Op.new(:op<bind>,
     QAST::Var(:name<$_>, :scope<lexical>)
     QAST::IVal.new(:value(0))
   }
   if
       # is the Opcode a bind QAST::Op ? 
       nqp::istype($ast, QAST::Op) && nqp::iseq_s($ast.op, 'bind') &&
       # is bind first an variable named $zero ? 
       nqp::istype($ast[0], QAST::Var)  && nqp::iseq_s($ast[0].name, '$zero')
       nqp::istype($ast[1], QAST::IVal) && nqp::iseq_i($ast[1].value, $int-matched )

   { ... }




  

   ATM 0






  CC<~$/>


	self.define_slang('ATM', ATM::Grammar, ATM::Actions);


By default $~ATM


To implement our ATM grammar, we have the C<ATM-Grammar> grammar, which a special class. As a
first approximation, think it as a way of implementing a recursive descendant
compiler. Each method of a grammar is a way to recognize a production. There is
no separate lexing layer. But lexing methods  are generally introduced with the
C<token> keyword while parsing method usually are by the C<rule> keyword.
Once a production is recognized, the associated homonymous action method is called in
C<ATM-Actions>

   grammar ATM-Grammar {
     ...
     token int  { \d+ }
     ...
   }

   class ATM-Actions {
      ...
      method int($/) {  ...   }
      ...
   }
   
In the C<.int> action method we need to generate the AST for
C<  nqp::istype($ast, QAST::IVal) && nqp::iseq_i($ast.value, ~$/); >.
C<$/>, the parameter of the C<.int> method is the match produced by the
<.int> token. We are only interested by the string matched so the C<~$/>,
which is equivalent to C<$/.Str>


C<  nqp::if(nqp::istype($ast, QAST::IVal), nqp::iseq_i($ast.value, ~$/)); >.

The code to generate the AST for  C< nqp::istype($ast, QAST::IVal) > is:

    QAST::Op.new(:op<istype>,
        QAST::Var.new(:name<$ast>, scope<lexical>),
       QAST::Wval.new(:value($*W.find_sym(['QAST', 'IVal'])))))

=head2 Another slang to the rescue

Or, are we going further in Alice rabbit hole.
Nope,