Trac'ing tests status

[Moritz Lenz]

Allison Randal wrote:
> Will Coleda wrote:
>> We should be following rakudo's example of not closing tickets for
>> which tests could be written.
>> 
>> We could abuse the existing trac fields we have to help track this
>> status, or we could add a new "Tests" field, with "needs, has, can't,
>> <null>"
>> 
>> Moritz has suggested we should additionally track the name of the test file.
> 
> I'd rather see a new ticket created for adding the tests,

And who creates that ticket?

>  with a reference to the closed ticket.

... and the other way round; otherwise you don't know if tests exist by
looking at the closed ticket. So even more overhead.

> It does take a little more thought to 
> phrase the second ticket in what exactly can be tested, but that little 
> bit of effort turns it into a task we could hand to a new contributor 
> instead of requiring a full understanding of the original ticket.
> 
> So, add a Type 'test' in addition to 'bug', 'todo', 'cage'. The name of 
> the test file makes the most sense in the description, where you can 
> list multiple files, explain whether the tests need to be added to an 
> existing file or create a new file, and explain where similar or related 
> tests can be found as examples.

Somehow this sounds to me like more work than necessary; but then again
I mostly write tests for Perl 6 stuff, which is often easy: the bug
reports mostly contain a piece of code that misbehaves, turning that
into a test is no rocket science.

So I see two possibilities: either the parrot bugs are much harder to
test for - than the idea of giving them to newbies is moot. Or they are
simple to test with some code snippet already provided - then it's easy
to turn that into a formal test, no need to encapsulate it into a
separate ticket.

Is there enough middle ground between those two possibilities to warrant
the extra work flow effort?

Moritz