The Parrot is dead. Long live the Parrot?

Andrew Whitworth wknight8111 at gmail.com
Wed Feb 13 01:05:51 UTC 2013


On Tue, Feb 12, 2013 at 11:39 AM, Reini Urban <rurban at x-ray.at> wrote:
> imcc -O1 removal was nonsense, only -O2 was broken, and it was trivial
> to fix for me in one hour. but since someone just removed the whole
> imcc optimization infrastructure I was hesitant and horrified to bring
> it back in. parrot is a compiler, how would you argument that removing
> compiler optimizations on pir level helps the product? it only helps
> the managers who do not understand compiler optimizations.

I removed the optimization framework from IMCC. It was a good decision
to make. Much of the code was dead (not called from anywhere), and
coupling with the rest of Parrot was far too high. The goal was to
separate IMCC from the rest of libparrot. This is still a good goal. I
would do it again.

Parrot is not a compiler. It is a VM. IMCC is the compiler and it is
not even a good one. The two should never have been coupled together
like they are.  Like the JVM, Parrot should take compiled bytecode
files to run. IMCC (like javac) and other frontends should output
bytecode for Parrot to run. Ideally, IMCC should be deleted entirely
because it is garbage, and other frontends or compilers (like Rakudo,
Winxed or NQP) should be modified to output bytecode directly. We were
not too far from that goal but it was a lot of work to get as far as
we did.

> jit removal was non-sense and almost killed us. it should have been
> easy to fix. it was not just disabled, it was completely removed,
> without any chance to bring back in.

I removed the JIT too. I would do it again. It was a good decision.

The JIT was bad, not maintainable, not extensible, poorly implemented
and based on a bad design. The JIT implementation required every
opcode in parrot, all ~1200 of them, to  be implemented multiple
times. One for every single platform that the JIT targeted. You want
PowerPC support? Implement all 1200 opcodes and their variants. You
want x64 support? Implement all 1200 opcodes and their variants. You
want ARM support? Implement all 1200 opcodes and their variants.

ALL OF THIS WORK, and the JIT didn't support even basic optimizations,
tracing, inlining, type inference, type specialization or other
improvements. Not only were they not supported, they were effectively
impossible to add later. JIT is not good just because you spell it
J-I-T. JIT isn't good if it only turns bytecode into machine code with
similar performance characteristics. JIT is good ONLY when it makes
things faster and better through enabling optimizations. The old
Parrot JIT did not do this, could not be ported to other platforms,
and was garbage.

> parrot made significant progress last year, just lorito not.
> you are destroying it now in shere suicidal tendencies again just because a
> few outsiders badmouthed it? threads and the generational non-blocking
> gc were a dramatic progress and the endresult is that people now
> believe our threads are broken and jvm threads are better??? hell

Our threads are still experimental, and there are some severe
limitations. For instance, cross-thread data cannot be modified like
in the original design. I do not think it will be hard to add this
feature, but it is not possible now.

> I'm all for fixing the management problems, which led to disgust at perl6.
> killing all the languages, adding strange anti-perl6 policies, killing
> bytecompat, ...

The management problems which lead to killing all the languages, and
which lead to the anti-perl6 environment are gone now. The damage is
already done. We're not going to get them back just because we promise
to be nicer. WE'VE BEEN DOING THAT FOR TWO YEARS. Words and smiles
aren't enough to bring the two projects back together. The damage is
done. The friendship is broken. Rakudo is on a one-way ticket to other
VMs and honestly, *it's better for them to do it*.

They didn't just leave because we were being anti-social. There were
and still are technical reasons to leave. The anti-social parts didn't
help either.

I've been fixing and removing garbage, cruft and disease from Parrot
for over 4 years, and we are STILL HAVING THIS CONVERSATION about what
emergency measures we need to take to get it to stop being filled with
garbage, cruft and disease. We are still talking about the emergency
things we need to do to make Perl6 happy. And guess what? The things
we're talking about are things we were talking about doing 3 years
ago, and things that we have been doing all this time. We've been
removing things that Rakudo doesn't need, and we've been optimizing
things and we've been changing bad designs and bad interfaces. We can
keep fixing these things like rats running on a wheel.

In a year or two maybe, when we finally have fixed all of the problems
that Parrot had in 2008 (if we have more active developers than we've
had in months, and if we've all been perfect and haven't shoveled any
new crap onto the pile since then) Parrot will be a decade behind the
capabilities of the competition. How does that make Parrot a
compelling alternative to either of those? Rakudo is going to come
rushing back just because Parrot is less bad than it was? Other
languages are going to rush to us too, because we're less bad than the
last time they looked?

We have to ask ourselves a serious question: What is Parrot? What is
the goal of it? What is the dream that lead to it's creation? We
wanted a VM that was focused on dynamic languages and that allowed
interoperation between these languages. We wanted a world where we
could add features to one interpreter (threads, JIT and AOT,
optimizations, analysis, exceptions, bytecode, native call, etc) and
all languages could benefit from those. We could build up a large
ecosystem of modules and libraries, and all languages could share them
and grow at a faster rate than any one language could grow by itself.

Guess what? It's 2013 now and such products already exist: .NET and
JVM. On a .NET VM (or Mono) I can run IronRuby IronPython, JScript,
C#, F# AND Perl6 (Niecza). I can create libraries in each, reference
them together into a single project and run them together. It's not
perfect, but it's possible. In 2013, Parrot only has compilers for one
of these languages, and interoperation wouldn't be possible even if we
had a second.

I'm not advertising .NET or JVM, but the fact remains that they are
the competition and they are eating our lunch.

If you fix a problem in Parrot, you break the workarounds in NQP and
Rakudo, and that's going to make the Rakudo folks even less happy. If
you try to change something or remove something, you're going to break
the thousands of unit tests which prove that Parrot is broken and that
Parrot stays broken. There's inertia to keep it broken, and there
aren't enough developers around to un-break it in a timely manner (and
fix all the problems that fixes bring). And even if a few brave souls
do keep working on it, by the time you clear out all the garbage,
you're going to be even further behind the competition.

> rakudo will be happy, if they have a feature veto, a policy veto and
> if they see
> maintainance and performance improvements in parrot.

The only benefit Parrot has right now for Rakudo is that most of the
biggest pain points have been resolved and most of the workarounds are
already in place. Rakudo works on Parrot without any additional
effort, where other VM backends require some implementation to
complete. When those alternatives are complete, Parrot will have no
more advantages. Technologically, Parrot cannot compete.

Parrot was NOT designed with Perl6 in mind (even after the time that
the Perl6 spec stopped changing so much). Parrot was designed for some
mythical combination of languages which don't exist or which we don't
have compilers for now. If you want to rip out all the stuff that
Perl6 doesn't use, doesn't need, and doesn't do exactly what Perl6
needs, you're going to "rm -rf *" the repo and start over.

> I can only fix stuff which I am allowed to fix. if managers above me rather
> act suicidal I stay away from this mess.

We don't really have a management structure right now. As far as my
opinion counts, you are welcome to make any changes that you want to
make. There are no code reviews. If you want to fix something, do it.

I'm sorry to sound so negative here. I've been reflecting for a few
weeks and have reached some conclusions that I'm not happy with. I'm
not suggesting that anybody stop working on Parrot, but we have to be
a heck of a lot more realistic about what our goals are and take a
long hard look at what we hope to accomplish. The old philosophical
points that set Parrot apart from other VM projects don't
differentiate us anymore. Maybe that's where we need to start our
conversation.

--Andrew Whitworth


More information about the parrot-dev mailing list