sound distribution practices

[Jonathan Worthington]

On 4/7/2012 19:11, Allison Randal wrote:
> Okay, instead of just complaining about this, I want to do something
> about it. The Rakudo packages are so fragile on Debian, that they need
> special constraints to make sure the Parrot packages are held back until
> the Rakudo packages are updated.
>
> So, why is Rakudo so dependent on one specific month release of Parrot?
> Parrot isn't moving so fast that the APIs are all changing every month.
> Is NQP or Rakudo not respecting API boundaries? Are there some libraries
> currently shipping in Parrot that should be in NQP, Rakudo, or broken
> out into their own distributions instead?
Easiest way to analyze it is just git log on 
build/tools/PARROT_REVISION. Here I've done a very rough categorization 
of the last 20 entries (which takes us back to October).

Some of them are to take advantage of Parrot feature additions:

     bump PARROT_REVISION to get separate unlink and rmdir primtives

     use parrot's new Hash.update method to speed up method cache generation

     bump parrot revision to one that has the is_inf_or_nan opcode

     use parrot op to switch to the profiling runcore

So not bumping would mean Rakudo forgoing making user-facing improvements.

Some are to get Parrot fixes:

     bump parrot revision to get MacOS build fixes

     Bump to a Parrot version with a fix for ByteBuffer segfaults, which 
caused programs using Buf to sometimes segfault.

Again, there are Rakudo user benefits to following this.

Some are due to chasing API changes in Parrot:

     Update to new version of getprop op

     Bump parrot revision to kill_props_vtables marge commit.

The getprop op changed at a PIR level. No guts poking here, just a (for 
the better) change. I should add that when I say "chasing" in reality 
Parrot folks have supplied the patches here, and it's been managed smoothly.

One is due to getting a PAST improvement:

     bump parrot revision to get directaccess support, use directaccess 
for most var lookups

We're currently in the process of redeveloping PAST so it's written in 
NQP, can do native types better, is more memory efficient and so forth, 
and that work will be in the NQP repository, which would have avoided a 
bump of this nature - but it was just one bump out of 20.

Which leaves the biggest group, which are a combination of a policy of 
targeting the latest Parrot in releases and wanting to have those using 
Rakudo from git testing against latest Parrot after sizable changes, so 
we can have faster feedback if there are problems, which would seem 
desirable for both Rakudo and Parrot devs.

     bump parrot revision to 4.2.0 release

     bump parrot revision
     just to get some better testing of newer birds, no specific reason

     bump parrot revision to something after the cont_reuse merge

     bump parrot requirement to 4.1 release

     bump parrot revision to get some testing

     bump PARROT_REVISION to 3.11 release

     bump parrot revision

     Bump PARROT_REVISION

     Bump to Parrot 3.10 release.

     Bump to latest Parrot (not quite release yet) in order to get more 
testing against it.

     bump PARROT_REVISION to after the green_threads merge to get some 
more testing

     bump PARROT_REVISION

Granted some in this category aren't explicit about the reason why, so 
there's some error in the analysis here.

> Rakudo should be able to run on any Parrot release within the past year
> (at least). The fact that it can't is a sign of poor software. We need
> to figure out what's broken and fix it. I'm happy to help. Highly
> motivated too, as it would remove a regular source of irritation.
There seems to be a few things going on here. One is that we don't 
differentiate Parrot revision requirement updates that are "must have" 
from those where we're bumping it for development reasons rather than 
because it's a hard dependency. Another is that Rakudo's feature needs 
often drive feature additions to Parrot and drive some bug fixes too; it 
seems inevitable that Rakudo will want to quickly take advantage of them.

Generally, I think things are currently more -Odevelopment than 
-Odistribution. I agree that makes things harder for those working on 
distributions, and I'm interested to make things better - I want it to 
be as easy for people to get their hands on recent Rakudo releases, and 
having it packaged for distributions does help with that. On the other 
hand, it's also an accurate reflection of the current state of both 
projects: in need of development work, and optimizing for it.

Also just to check - are you packaging Rakudo compiler releases, or 
Rakudo distribution releases (which are the ones we more intend for 
packaging)? Mostly asking due to the timing of the email; we didn't do a 
distribution release for March (so last interested one to package was in 
Feb), and the next one is due this month.

Thanks,

Jonathan