Recent changes impeded building and testing on smaller boxes

[Patrick R. Michaud]

On Sun, Sep 26, 2010 at 07:32:38PM +0200, Nick Wellnhofer wrote:
> The only big problem at the moment is the memory consumption during
> a Rakudo build. Building core.pir uses about 240MB plus GC overhead,
> so this will cause swapping on a 256MB machine. I think the build
> process could be changed to consume less memory but I'm no expert in
> this area.

I've only come up with two possibilities for reducing memory usage,
neither of which are trivial (but both of which are worth pursuing):
   1.  Make the overall compiler more efficient
   2.  Separate the core.pm compilation into smaller units

#1 is difficult to do at the moment because we lack profiling tools 
to help us determine where things are memory inefficient.  For example,
it's possible (even likely) that no-longer-needed large data structures
are being held by a single context register that could otherwise be
nulled out... but afaik there's not an easy way to detect this.

#2 is difficult because Perl 6 expects all of the core library routines
and classes to be contained in a single lexical scope; breaking those
into separate components means we need some mechanism that can stitch
the pieces back together into a shared lexical scope, either before 
being passed along to the PIR compiler or dynamically at runtime.
And Parrot doesn't have good support for dynamically modifying a
lexical scope at runtime, although there are some very hacky workarounds.
Also, we have to deal with a few circular dependencies in the core.

Pm