Euler Project Problem #4 Benchmark on Parrot 0.2.0 - trunk (r42903)

Patrick R. Michaud pmichaud at pobox.com
Sun Dec 6 19:18:07 UTC 2009


On Sun, Dec 06, 2009 at 08:06:50AM -0800, Geoffrey Broadwell wrote:
> On Sun, 2009-12-06 at 01:33 -0800, Jonathan Leto wrote:
> > This one is all over the place and trunk is not looking very pretty.
> > http://gist.github.com/250139
> 
> 1. Make sure HEAD is in the same grouping as the fastest time, for all
> benchmarks we can find.  
> 2. Make sure HEAD has the fastest time on all of our benchmarks.
> 
> Right now I think we should be focusing on #1, both because #2 is too
> far ahead of where we are now, and because violations of #1 will tend to
> tell us when we've found a particular use case that needs more immediate
> attention.

This thread and discussion bothers me a fair bit, because it seems to
imply that we should be focusing our limited resources on optimizing
Parrot for code that is *completely unlike* anything our target HLLs are
likely to be generating anytime in the near future.  

Given Parrot's other limitations, there is almost no way that
we could get from PHP, Perl 5, Perl 6, or Python source to the PIR
code that is being measured in this benchmark (except perhaps by
HLL compilers that are specifically tuned to this purpose).  As such, 
these results are completely unrepresentative of HLL performance
on Parrot.  Indeed, one reason why older Parrots are able to perform 
better is because they were able to ignore the very features (and
overhead) that have been added to support HLL development.

Beyond that, I do not agree with any premise that HLLs will benefit 
from the optimizations this sort of analysis can achieve.  I 
will even go so far as to say that many of the HLL performance
and code difficulties we see today are *precisely* because of 
past efforts to optimize Parrot around unrealistic PIR snippets 
like these.  What Parrot needs to do instead is to analyze,
benchmark, and optimize around the type of code that HLLs will 
actually generate.

So, if there are Parrot developers out there who are only interested
in optimizing benchmarks like these, then I'm fine with those
individuals pursing that interest.  But generalizing to say
that Parrot development as a whole should focus on optimizing
_away_ from today's HLL developer needs bothers me a lot.

Pm


More information about the parrot-dev mailing list