Parrot's build process and "make install"

[Patrick R. Michaud]

At yesterday's #parrotsketch we briefly summarized where things
are with Parrot's build and install process, and I indicated
that I'd summarize a list of things to be done and throw a lot 
of tuits at the problem this week.

The list of things to be done summary is below.  But in thinking
about the issues further I've decided that I'm probably not going to be
able to throw as many tuits at is as I had originally thought, so
I'm hoping others can step in and pick up the tasks.

(This might mean the build/install refactors don't make it into the 
1.4 release; I'm okay with that if others are.  If others aren't 
okay with it not being done before 1.4, then "Thanks for volunteering!" :-)

So, where things stand now:  First, there are quite a few tickets
in trac dealing with overall build, configuration, and install
issues.  I reviewed this list, closed a few, and placed the
remaining tickets in the "install" component.  You can likely
see the full list via the following query:

  https://trac.parrot.org/parrot/query?status=!closed&component=install

I particularly recommend TT #649 -- Andy Dougherty has collected
a useful summary of parrot config and install issues there.
Indeed, I tend to add a +1 to everything Andy has written on this
subject.

Where things need to go from here:  At PVMW, YAPC::NA, and in
conversations on #parrotsketch, there seems to be widespread
agreement that Parrot's "make" target really should create 
something that looks a lot more like an install (or install-dev) 
tree.  Then the "make install" and "make install-dev" targets end
up doing something more like a copy than what they currently do
(which iiuc is "build new (untested!) executables and install those").

In particular, I feel fairly strongly that the parrot executable
that gets built should _not_ be reading any of its files that it 
uses from their original source locations.  For example, a parrot 
executable should never read things from runtime/parrot/library/
in the build tree.  Instead, just like an installed parrot,
the newly should use (versiondir)/library/ but have (versiondir)
point to the pre-install image of the build.

It also concerns me that "make test" doesn't really test
a parrot installation, or even anything that looks close to one.
Instead it tests the execution of things as they exist in their
build locations.  Switching "make test" to test something that looks
more like an install would help us avoid problems arising from
missing/misconfigured components in the install tree.

One of the benefits of a refactor like this should be that we no 
longer need separate "build" versus "install" parrot_config values, 
as we have now (TT #649).  Instead the config values for most libraries
and other support files would remain the same, with only the
prefix changing for installed versus build-tree copies of
the files.

One approach might be to rearrange the structure of Parrot's
source tree itself so that it mimics the structure of an installed
Parrot.  IMO this should come about in a second round of refactoring,
if at all -- many of the existing build tools depend on the current
source tree structure and changing those would seem to be "too big
a leap" for one refactor.  But I wouldn't object to someone trying
that approach if it seems easier.

There are some other issues that also have to be addressed in
order for HLLs like Rakudo to effectively be able to compile
and run from an installed Parrot -- I'll summarize those in
a separate email or ticket, as we'd _really_ like those to be
addressed for 1.4.0 .  This message is really intended to provide
a basis for anyone that wants to work on substantially cleaning
up and improving Parrot's overall build/install process itself.

And now that I've studied up on the issues a bit further, I'll
be glad to provide quick answers to anyone who might chose to
work in this area.  My answers might end up being incorrect
answers that are quickly overruled by the Parrot architect,
but at least you'll have quick answers to be working from.  :-)

Thanks!

Pm