random stuff that needs doing on the road to Parrot 6

notes.md

Easy(-ish) stuff

• resurrect tools/build/nativecall.pl (removed in Parrot 2.1.0) or port tools/dev/nci_thunk_gen.pir to Perl5 ( I've started with the latter, but progress is slow as it's boring )

  this is desirable as nci_thunk_gen.pir pulls in TGE via compilers/data_json

• put the axe to runtime/parrot/library

• migrate ops2c to NQP proper (or even Perl6)

  it should be feasible as the files generated for NQP dynops can be checked-in the same way the Parrot ones already are

• pull in NQP and Rakudo dynops into Parrot core

  the final goal would be a single dynamic library instead of the current directory full of dynamic extensions, most of which need to be loaded anyway for Rakudo to function

• add --target=pbc to the nqp executable so a 2-stage module build is no longer necessary

Hard stuff

• bacek in #parrot:

    Coke, start with "Class PMC" and related ops. It will be a huge cleanup of totally useless code.
    And yeah. Kill MMD with fire
    Just kill it
    And remove all of PCC nonsense
  

• rurban in #perl6:

    You should inspect the calling convention and undo the nci method madness
    ops2c should be reverted to the perl5 lib, so that nqp can go away
    or use nqp-p6 (i have a branch)
    and use bacek's llvm ops2c
    which jit's the ops.
    So I guess you'll need nqp for the jit
    I fixed llvm integration already to use static or shared llvm libs
    And removing imcc -O1 was also not helpful if you want a faster rakudo
  

• whiteknight on parrot-dev:

    You absolutely must complete the work we started of trying to have
    Rakudo generate bytecode natively instead of using PIR as an
    intermediary. The effects on compilation (and script interpretation)
    performance is debilitating. IMCC needs to be taken away and burned in
    a fire. NQP needs to be updated to output Parrot bytecode directly.
    
    You'd have to rip out CallContext and the entire PCC system, and
    replace it all with NQP's dispatcher and argument binder (which,
    again, would need to be rewritten to account for this missing
    infrastructure). If you can do this, you'll see a nice performance
    bump in every single method invocation, which will make for much
    speedier programs and much snappier compilation times of the Perl6
    core.
  

• mls on parrot-dev:

    IMHO the problem with the current implementation is that it creates
    three PMCs for every call:
    - the call context, which holds all of the registers
    - the return continuation
    - in most cases a lexpad, which is in fact not needed for rakudo
    
    The profiling I did some time ago indicated that that's mostly why
    calling a function is so slow. A long time ago the call context
    was not a PMC, making it a PMC may have been the code somewhat
    cleaner, but I think it was a mistake performance wise.
    
    The return continuation could also be integrated into the call
    context, I once did that as quick hack and got a 16% speed
    improvement AFAIR.
    
    The lexpad should be optional instead of mandatory. For rakudo,
    the lexinfo and the call context is all that's needed for the
    lexical lookups.
  

Note about "migrate ops2c to NQP proper". nqp-rx and NQP have a fundamental difference that complicates this a fair bit.

nqp-rx attempted to restrict itself to pure Parrot ops as much as possible, and created code that didn't use any custom ops or libraries. Thus one could use nqp-rx to write programs that would generate PIR that could run standalone (i.e., when nqp-rx runtime isn't present). In other words, nqp-rx attempted to make sure it had very minimal runtime requirements.

NQP generates code that fully expects NQP opcodes and libraries to be present; you can't simply take the PIR generated by NQP and run it without having the NQP dynlibs available as well.

Rakudo has even larger requirements, in order to run a precompiled Rakudo program you have to have the full Rakudo libraries available. More than that, you have to have the exact version of libraries that were in place at the time the code was generated (long story).

So, if rewriting ops2c in NQP or Rakudo, just know that you'll have to carry around the corresponding runtime components as well.

Pm

"pull in NQP and Rakudo dynops into Parrot core"

I'd prefer to focus primarily on NQP dynops. Even if Parrot decides to focus primarily on Rakudo, there doesn't seem to be a compelling reason for Rakudo dynops to appear in Parrot core.

Note that many NQP dynops only make sense when sixmodel is present.

Pm

@pmichaud:

So, if rewriting ops2c in NQP or Rakudo, just know that you'll have to carry around the corresponding runtime components as well.

I don't believe that's a problem:

$ make help | grep bootstrap
  bootstrap-ops:     Generate C code from .ops files. Requires already built parrot.
  bootstrap-nci:     Generate C code for NCI. Requires already built parrot.
  bootstrap-prt0:    Generate prt0.pir. Requires already built parrot.

The story would change from Requires already built parrot to Requires already built NQP. As Rakudo is Parrot's only remaining customer, there's no practical benefit in having a Parrot that can bootstrap without NQP.

I'd prefer to focus primarily on NQP dynops. Even if Parrot decides to focus primarily on Rakudo, there doesn't seem to be a compelling reason for Rakudo dynops to appear in Parrot core.

Note that many NQP dynops only make sense when sixmodel is present.

Agreed and so noted. The reasoning behind including them in Parrot proper is ultimately to ship just a handful of files instead of a whole ecosystem as is done right now, and one step on that road is moving the dynops and dynpmcs to core. I do agree that it makes sense to keep Rakudo separate.