Parrot "standard libraries"

[Geoffrey Broadwell]

This message is a reply to collected objections to increasing the number
of modules in the Parrot core set; I've sent another discussing how to
decide which modules would be good candidates to add.
-----

On Thu, 2009-07-30 at 07:07 -0400, James E Keenan wrote:
> 1.  Discussion of the benefits of adding any particular library to our 
> "standard set" should also include any disadvantages or risks that that 
> addition may pose.

Certainly.  My hunch is that there are a decent number of valuable
libraries for which the risks are few, but that will merely make the
discussion go quickly.  :-)

> Potential downsides would include:  More complexity in configuration?

If this becomes an issue, there may be value in separating the
configuration into two separate conceptual stages -- core config and
module config -- which we can treat differently in ways that will be a
net win for complexity management.  (Note that the user should only be
expected to run one configuration script -- both stages would be
launched by the same script.)

However, I think this is a bridge to cross when we come to it.

> Slows down the executables?  Bigger memory footprint?

Most modules should only affect performance and memory footprint when
they are actually used.  This is probably only a worry if a module
cannot be implemented properly without affecting Parrot's performance or
memory usage even when *not* in use (for example, the penalty that perl5
pays for being compiled with threads support, even if threads aren't
actually used at runtime).

> Perhaps we should set some minimum criterion of benefits over risks that 
> a library would have to meet before it is considered a standard part of 
> this virtual machine.

See my other message.  :-)

> 2.  And if we were to develop such a benefit/risk criterion, we should 
> probably apply it to the libraries for which we *already* probe during 
> the 'config::auto' steps.  Most of those steps were added before I 
> joined the Parrot project.  The fact that we probe for them indicates 
> that at some point in the past someone (with a commit bit) made a case 
> for including them, but what risks they might entail and whether they 
> bring a net benefit to Parrot I simply cannot say.

Sure, that's only fair (though do note that removing any of these would
require a deprecation cycle).

On Thu, 30 Jul 2009, Michael Peters wrote:
> Another downside is that if you pick a version of the library to
> interface with, it might be newer or older than the version of that
> library shipped by an OS distro. So they'd only be able to use a
> version of Parrot that uses that same version of libsqlite.so (just
> an example) that they are using for their other systems. And it gets
> more complicated the more libraries are used. I don't know how hard
> this will be in practice, but it's something to consider.

I'm not sure that this is any different from the version dependency
issues of separately packaged modules, except that if we force every HLL
to wrap libsqlite themselves, rather than doing it once in the Parrot
core, the various HLLs are likely to choose *different* versions of the
library to wrap, and then the user will be in a dependency hell of a
whole new order (yes, Perl 6 explicitly tries to deal with this in the
HLL case -- I'm not sure that helps when trying to load the .so's,
though.)

In addition, if we only have to spend the energy to make a wrapper
compatible with different versions of the library *once*, we (and the
distro packagers) are more likely to expend the effort, and the user
will be happier for it.

On Fri, 2009-07-31 at 10:37 +0400, Richard Hainsworth wrote: 
> I dont think it is possible to decide upon a base set of libraries to 
> probe for. What is essential for one problem space is irrelevant for 
> another. Compromising and including both is not an elegant solution, 
> leading to bloat and increased startup times.

This seems to assume that there will be a significant runtime startup
cost for merely having modules available, even when they are not used.
I think this is vastly more likely to be the exception than the rule;
there should only be a cost to bring in a module with 'load_bytecode',
not to merely have it sitting quietly on disk unused.

> I suggest a configuration file that lists the libraries and dependencies 
> to be probed for when parrot starts up. If the configuration file also 
> provides helpful error messages when a probe does not work, that will 
> aid when dependencies need to be added to a system.

Why would you want to probe for libraries and dependencies at every
startup?  That's a plugin system, and it's not what I'd suggest at all.
Certainly when a Rakudo user says 'use Foo:from<parrot>;', Parrot will
have to check if Foo.pbc exists on disk -- but there's no reason Parrot
should also check for {Bar,Baz,Quux}.pbc until they are actually
requested.

> I suggest that NO base libraries are expected by Parrot.

Parrot *itself* may need a few, such as Unicode libs, but beyond that
I'd say stripping Parrot to the metal is a job for distro packagers.  We
shouldn't force the HLLs to go completely without standard modules by
default; distro packagers can separate them out easily enough if they
feel the need.

> Parrot should only have a pluggable means for testing for libraries via 
> the configuration. All libraries would then have to conform to some set 
> of standards.

That's not a bad idea, to help deal with James Keenan's concerns about
growing complexity.

> A clear policy by the Parrot development team about what a base library 
> should look like and where it should be located will allow for 
> flexibility and the ability for developers to pick and choose.

Also a good idea.


-'f