Proposed critical step in cutting 1.0 release (or any major release)

[Christoph Otto]

Patrick R. Michaud wrote:
> On Mon, Mar 02, 2009 at 12:22:23PM -0800, chromatic wrote:
>> On Monday 02 March 2009 09:41:51 Patrick R. Michaud wrote:
>>
>>> I suggest that we add a "critical" ticket for each of
>>> the major releases whereby we verify that the major HLLs
>>> can still be built against the code being considered for
>>> the release, or at least that we can positively identify
>>> that any problems are in the HLL code and not something
>>> that requires a Parrot internals fix (as was needed for
>>> r37061).
>> We also need tests for *every* bugfix for HLL breakage.  Every time our 
>> existing test suite doesn't catch this problem, we know it's incomplete.  I 
>> didn't see a test added for r37061.
> 
> To be honest, I don't think any of us know what caused the 
> r37061 bug in the first place to be able to formulate it into 
> a test.  Like you, I'm presuming that all of the other 'make 
> test' scripts worked fine, and it was only when trying to compile
> Rakudo (perl6.pir) to perl6.pbc that we saw the bug.
> 

Correct.  The bug was not exposed by Parrot's make test, but only by Rakudo's 
build.  I'll close TT #388 once there's a test in Parrot to make sure this bug 
doesn't resurface.


Better HLL testing strikes me as a good idea.  However, it will also 
significantly increase the time it takes to test a change, especially as other 
HLLs are developed.  Is there a good rule of thumb for which tests should be 
run when testing a change or bugfix, or is the advice "run rakudo's spectest" 
every time?