PDS aftermath: .nqp programs in Parrot core

[Andrew Whitworth]

On Mon, Jan 31, 2011 at 9:30 AM, jerry gay <jerry.gay at gmail.com> wrote:
> the concern with parrot supporting the new nqp is that any tools we
> write using nqp can be easily ported to any other vm. however
> altruistic, portability from parrot to other backends is not currently
> one of parrot's goals. in fact, chromatic, myself, and others believe
> it may be harmful to parrot's future to support portability of
> nqp-based parrot tools to other backends. parrot is simply not in a
> position to compete with more mature virtual machines yet, and the
> risk that the new nqp's portability poses is significant.

We are potentially losing, in time, an exclusive relationship we've
had with a very close sister-community. This much is true and it can
be a little bit distressing to think about. We aren't losing Rakudo or
NQP as users of Parrot, we are only losing the exclusivity that comes
from them choosing not also to explore and support other VMs in
addition.

What we should not do is enter into some kind of panic-induced
protectionism, and start declaring that people are "with us" or
"against us". This isn't the case and isn't helpful.

It's in the best interests of Rakudo to spread out to support other
backends. First, if Parrot never improves and never becomes a viable
alternative, Rakudo needs to do what it can do to survive without us.
Of course, I don't think Parrot's road ends in such dismal failure,
but Rakudo does need to take steps to protect itself and we all need
to appreciate this. Keep in mind that Parrot has made a number of
decisions in the past to *introduce* semantic disparities between
Parrot and Perl6, because it was felt that not doing things the exact
"Perl6 way" would help entice other HLLs to start targetting Parrot.

We support other HLLs besides Rakudo Perl6, and they support other
backends besides Parrot.  That said, Parrot itself doesn't necessarily
need to be supporting other VMs as well. That certainly is outside our
purview.  Features from Rakudo and NQP which we adopt and merge into
Parrot will be done natively and won't include portability to other
VMs. NQP and Rakudo can provide their own compatibility wrappers for
other platforms. As I will describe below, this has very little impact
on languages which we use inside the Parrot repo for our own tools and
utilities.

> none of this prevents parrot from continuing to use the existing
> nqp-rx. it is likely that no change to parrot's nqp-based toolchain is
> necessary, however small changes may be desired (or required) to make
> clear the distinction between the new nqp and nqp-rx. parrot may not
> even need to fork nqp-rx, if the nqp team agrees to give over
> ownership of nqp-rx to parrot, or agrees to give parrot folks liberal
> commit rights to nqp-rx. to my knowledge nobody has approached the nqp
> team with a proposal similar to those i mentioned above. in the
> unlikely event that the nqp team is not receptive, a fork of nqp-rx is
> simple, straightforward, and justified.

If we're going to be writing some of our most fundamental tools in a
particular language-on-parrot, we are going to want to have that
compiler available directly in the repo, or have it developed under
the Parrot umbrella organization on Github and snapshot'd into the
Parrot repo. Such a language should likely be small, not require a
huge runtime library (if any at all), should enable easy access to all
the features of Parrot (especially necessary for testing those
features). For clarity, I'll call the languages that we provide in the
repo and use to implement our core tools, utilities, and libraries as
our "core" languages.

What language(s) is this? PIR is the only thing that really satisfies
all the requirements I outline above right now. It's ugly and nobody
likes it, but replacing IMCC with PIRATE will help. Also, once we have
Lorito and direct-to-bytecode HLL compilation, PIR would no longer
need to be optimized for parsability and can start to take on
higher-level language features like loops, proper if/then/else, and
other things. Also, even though we have rewritten some tools from PIR
into NQP, we still have a huge base of existing PIR code that would be
very hard to update to something else en masse.

NQP-RX doesn't belong to Parrot, but maybe could eventually if the
Perl6 people have no further use of it after the NQP migration. Also,
we can fork it. This would serve as a good starting point, but would
likely need to be modified significantly to suit all the needs of
Parrot. These modifications, over time, will lead to significant
divergence from the "real" NQP and Perl6 syntax and semantics. We
would probably want to rename our version to something besides "NQP"
to avoid confusion, and make clear the fact that it may be more than
just "not quite" perl-like eventually.

The new NQP I don't think will ever be acceptable for writing our core
utilities in, even after we've adopted 6model into core. The problem
is that we don't have any real control over it, and we can't be
expecting our lowest-level utility language which we use to implement
all our build tools, utilities, and libraries in to be written in a
language over which we have no control and which doesn't provide any
low-level interface to our VM. The fact that the new NQP will support
other backends is not an issue in this decision at all.

Winxed is another option. We also don't own it, but we could either
beg NotFound to make a gift or we could fork it. It satisfies all the
other stated requirements.

Parrot's repository should not contain any tools, utilities, or
libraries written in a language which is not itself provided in the
Parrot repo (with a *very* small handful of exceptions, including
Configure.pl and maybe the Python-based GDB pretty-printers). In fact,
many of these modules which currently do live in the Parrot repository
could likely move out of it, reducing the amount of stuff that needs
to be written in any particular set or subset of languages.

--Andrew Whitworth