[RFC][PATCH] Rename Rakudo's "spectest_regression" to "spectest"

[jerry gay]

On Sat, Oct 11, 2008 at 2:24 AM, Allison Randal <allison at parrot.org> wrote:
> Moritz Lenz wrote:
>>
>> jerry gay wrote:
>> A combined harness is much better in terms of reporting.
>
> Yes.
>
a combined harness is much easier now that we require T::H 3.
anybody with TH3-shaped tuits who could take a look at rakudo's
and/or parrot's test harness with an eye on combining harness results
would certainly be rewarded by increased karma and high praise.

>>> the tests we expect to pass reliably should be the default tests we
>>> run. we expect all spectest_regression tests to pass reliably. the
>>> default test target should always be named 'test'. it seems natural
>>> that we add spectest_regression to the 'test' makefile target.
>>> additionally, this would have possibly prevented the 74 failures
>>> post-mdd-merge, since allison didn't know about the additional test
>>> target in the makefile.
>>
>> well, if reading the README is too much even for our architect then we
>> shouldn't assume that anybody else does ;-)
>
> Another thing that would be helpful for languages in trunk is something like
> a TESTME file. It should briefly say exactly what steps a core developer
> should take to test that their changes haven't broken the language, and if
> failures are expected or all tests should pass. Also, anything strange like
> having multiple test harnesses running in sequence instead of aggregating
> the results in one report. (That one caught me on Rakudo's 'make test' too.
> I thought all the test were passing, and then found that the final "All
> tests pass" report was hiding earlier failures in a different summary.) The
> README is quite verbose and intended for people who want to use the
> language. Even after reading it, it's not straightforward to decide what to
> test and whether failures are relevant.
>
i'll try to address rakudo's documentation this weekend at pittsburgh
perl workshop.
this seems like a great task for new contributors, working alongside old hands.

> What would be really ideal is if core developers could just run 'make
> languagetest' in the repository root and get a single report of all the
> language failures, and know for sure that any failures are their
> responsibility. But, we're a long way away from that.
>
TH3 makes this a lot easier than previously.
~jerry