Memory needed to build Perl 6 on Parrot; problem, goal, incentive for solution

[James E Keenan]

1.  Problem.

Last week I successfully built Rakudo Star from source on two different 
Linux/i386 machines, one real, one virtual.  However, my attempt to 
build Rakudo Star on my iBook was unsuccessful.  It repeatedly failed at 
this point:

   /Users/jimk/work/rstar/rakudo-star-2010.07/install/bin/parrot 
src/gen/perl6.pbc --target=pir \
     src/gen/core.pm > src/gen/core.pir

I filed an RT for this: 
http://rt.perl.org/rt3//Ticket/Display.html?id=76828.

In the course of discussion in that ticket, I noted that I had been able 
to build Perl 6 on top of Parrot on that same machine at least as late 
as June 2008, when I led a workshop to do that at YAPC::NA in Chicago. 
Coke responded that the file where compilation failed was "the single 
largest compilation that occurs during the process, and requires a LOT 
of memory, and it's likely gotten bigger since the last time you ran." 
I have 256 MB of physical memory available on this machine, to which 
Coke responded, "Based on recent history, I think 1G is the minimum 
recommend memory for building rakudo (for that file in particular)."

2.  General Issue.

I recapitulate this discussion here because this is an issue that both 
the Parrot and Rakudo development teams will have to address:  What are 
the memory (and other) requirements for building Perl 6 on Parrot?

3.  Disposal of Red Herring; A Bit of History.

To be frank, one approach would be to say, "kid51, your iBook is six 
years old -- ancient in computer years.  You're the only one still 
submitting smolder tests for Darwin/PPC.  Go out and buy a new one and 
throw that G4 away."

You could say that, but of course I would not do that.  I am 
tempermentally the kind of person who keeps hardware running as long as 
possible.

But more to the point, this kind of computer is where we came in!  When 
the Parrot project was born nine years ago, this kind of computer -- and 
more importantly, computers with this size memory -- was state of the 
art.  It was what the original Parrot developers would have been using. 
  (ISTR being told at YAPC::NA::2002 in St Louis that Apple had given 
iBooks to "the Perl 6 development team."   They certainly seemed to be 
ubiquitous at that conference.)

So, if we were to turn away from building Rakudo Star on a box of this 
size, we would be saying, "We can't build Perl 6 on the machines on 
which we started out to write Perl 6."  To me, that's a very sub-optimal 
solution.

4.  Current Issues.

In the days since Rakudo Star was released, critics have taken Rakudo 
Star to task for (a) a slower start-up time than Perl 5; and (b) a 
slower total runtime than Perl 5.  We have acknowledged those concerns 
-- and have done so for a long time -- but have argued that we want to 
get things right before we optimize for speed.

But we've had much less focus on memory usage and apparently none on the 
amount of memory needed to *build* Rakudo (as distinct from the amount 
needed to *run* Rakudo).  My hunch is that most Parrot and Rakudo 
developers are, unlike me, early adopters of hardware that has come with 
vastly greater amounts of memory than mine, and therefore have never 
been bitten by the amount of memory needed to build Rakudo.

5.  A New Goal and Obstacles to Achieving It.

I think that lowering the memory required to build Rakudo ought to be an 
explicit goal of both Parrot and Rakudo.  *Do you agree?*

If so, then we have to prioritize efforts to reach this goal and herd 
our developer cats in the direction of a solution.  And, to complicate 
matters, we have to recognize that this will be exponentially harder 
because it will require changes in both Parrot and Rakudo.  Both will be 
difficult.  Parrot will have to shrink its memory footprint.  And Coke 
has suggested on IRC that the file in Rakudo which failed to compile is 
one of the biggest and most difficult in that distribution. 
Furthermore, the sort of changes that are needed are way out of my 
personal area of expertise.

6.  Incentive.

But perhaps I can offer an incentive!  I am prepared to offer a prize of 
US$200 (total), payable in cash or in beer, to the person or persons who 
enable Perl 6 to once again build on Rakudo Star on my iBook G4.

Stipulations:  We're talking about Rakudo Star here; not the monthly 
Rakudo compiler releases.  Cet. par., we'll assume that the solution 
lies partly in Parrot and partly in Rakudo and, hence, the prize will be 
shared.   The winner of the Parrot part of the prize will be decided in 
#parrotsketch.  The winner of the Rakudo part will be decided in 
whatever is Rakudo's equivalent to #parrotsketch.  And I will be the 
person who decides whether Rakudo Star has actually successfully built 
on my laptop.

Any takers?

Thank you very much.
kid51