Rakudo compilation times from 3.0.0 to now

[Jonathan "Duke" Leto]

Howdy,

Thank you moritz++, this is very valuable data.

Is there any way you can generate similar data for small, specific subsets of
the spectests? Such as, subsets for OO, String, Float and Integer performance.
That way, we can pinpoint which subsystems are having performance regressions.

I would love to help with the code to run and graph this benchmark data. My
goal is to automate the gathering and visualizing of performance data  and
possibly make it a standard feature in Jitterbug [0].

Duke

[0] https://github.com/franckcuny/jitterbug

On Tue, May 3, 2011 at 1:27 PM, Moritz Lenz <moritz at faui2k3.org> wrote:
> Hi all,
>
> so, I've done some timings of rakudo compilation times.
>
> A picture says more than 1000 words, so please take a look at the file
> in the attachement. In case some stupid filter strips it out, you an
> also find it here:
> http://moritz.faui2k3.org/tmp/rakudo-perl-2011-05-03.png
>
> The x axis needs a bit explaining: Since i couldn't be bothered to beat
> gnuplot into submission for plotting arbitrary labels on the x axis, I
> just numbered the versions:
>
> # in plot   rakudo    parrot   gc
> 1           2011.01   3.0.0    ms2
> 2           2011.02   3.1.0    ms2
> 3           2011.03   3.2.0    gms
> 4           2011.04   3.3.0    gms
> 5           2011.04-9-g7114f56  RELEASE_3_3_0-82-g1149c70  gms # not a
> release
>
> On the y axis you can find the compilation time starting from core.pm
> until rakudo is compiled to the bitter end (dominated by core.pm ->
> core.pir), divided by the number of lines in core.pm.
>
> I did three measurements for each rakudo/parrot combination, and took
> half of the difference of fasted and slowest as error bar, and the
> average as the center value. (No, not exact science :-)
>
> What you can see is that the compilation time is constantly increasing,
> only interrupted by the introduction of gms.
>
> I don't know if it's presumptuous to draw conclusions from 5x3 data
> points, but it seems that parrot is constantly getting slower, except
> when bacek installs a new GC.
>
> (Note that rakudo development was relatively slow in 2011, with much of
> the work going on in nqp at that time. So while rakudo changes can't be
> fully excluded as reasons for this slowdown, it seems unlikely that they
> cause the major part of it).
>
> Cheers,
> Moritz
>
> _______________________________________________
> http://lists.parrot.org/mailman/listinfo/parrot-dev
>
>



-- 
Jonathan "Duke" Leto <jonathan at leto.net>
209.691.DUKE // http://leto.net
NOTE: Personal email is only checked twice a day at 10am/2pm PST,
please call/text for time-sensitive matters.