Rakudo's branch to use nqp-rx

[Patrick R. Michaud]

On Tue, Nov 03, 2009 at 08:42:04AM +0100, Moritz Lenz wrote:
> Jonathan Leto wrote:
> > This is Rakudo's branch to integrate the new nqp-rx:
> > 
> > http://github.com/rakudo/rakudo/commits/usenqprx
> 
> Actually that was just the start, now the work continues in the "ng" branch.
> [...]

To be clearer -- the two branches have entirely different purposes.

The usenqprx branch was simply an experiment by Jonathan to see 
if nqprx could be used in place of the old nqp to compile Rakudo and
pass its spectests.  The answer ended up being "yes, and much 
more easily so than any of us had expected".  Jonathan achieved his
goal in just a few hours, and much of that time was spent exploring 
and testing as opposed to tracking down bugs.  The experiment basically
proves that code written for the old nqp can be made to work with
the new nqp with a minimum of hassles.  We're basically done with
the usenqprx branch, but are keeping it around in case we run into
difficulties on the ng branch.

The "ng" branch is basically rewriting nearly all of Rakudo's
internals to make use of the new features available in nqp-rx.  
Many critical components of Rakudo have been blocked while
waiting for these to be developed; and they go to the core
of the Perl 6 compilation process.  The specific features we 
need from nqp-rx (and that the old nqp and pge have little 
hope of ever providing) include:

    protoregexes
    contextual variables
    improved lexical lookup and initialization
    Perl 6 code blocks in regexes
    code assertions in regexes
    parameterized subrule calls
    parameters on regexes
    proper lexical scoping of regexes in outer blocks
    correct separation of roles between Cursor, Match, and grammars
    variable interpolation in regexes
    attributes in class declarations
    eventual longest token matching semantics

A quick look at Rakudo's docs/ROADMAP file will show how many
critical features were waiting on regex engine improvements,
and this is a primary reason why the regex engine and nqp
went through a significant redesign.  Of course, other libraries
and languages aren't required to use these new features -- the
old features tend to work just fine, as demonstrated by the
usenqprx branch.

Hope this helps,

Pm