Parrot "standard libraries"

[Geoffrey Broadwell]

This message is about figuring out what modules to include in the Parrot
core set.  I've sent another message responding to possible objections
to including any more than a completely minimal module set.
-----

Parrot has a primary goal to make it easy to bring up new dynamic
languages to the level of being productively useful.  This is greatly
assisted if the HLLs have access to a decent set of libraries, so that
end users can do cool things right off the bat.

There are two ways for HLLs to get access to "shared" libraries:

    1. Use libraries directly provided by core Parrot.
    2. Use libraries provided by other HLLs.

Ever since the major HLLs were split from Parrot, option 2 has become
weaker, in the sense that no HLL can *count* on another HLL being
available.  Thus HLL and HLL module implementers are more likely to
reimplement than tell their users "Want to use my spiffy new Cardinal
module?  First install Rakudo and these Rakudo modules ...."

That's fine as far as it goes; reimplementing templating systems and the
like is pretty much standard operating procedure even within a single
language's ecosystem, let alone across languages, and frankly I don't
think users would have it any other way.

On the other end of the scale, there are some modules that it's just
plain silly to make every HLL rewrite from scratch.  In my mind, this
group includes modules that are just thin wrappers over bog-standard C
libraries.  zlib is a perfect example of this -- it's the standard for
general compression, everyone will be using it, and there's no good
reason to make every HLL implementer hand-code the proper PIR to wrap it
as some sort of rite of passage.

This is even more true for zlib in particular because it's part of
Larry's conception of the "minimum necessary core" in order to get
access to an external module repository such as CPAN -- no one stores
source tarballs in uncompressed form.

(One could rightfully argue that each HLL will have different
colloquialisms/sugar that they will sprinkle over the raw C-style API,
and that's true.  But sugar is a higher layer than NCI thunks; even if
every HLL had wildly different sugar, they could all share the NCI
code.)

Between these two extremes is a pretty wide spectrum of modules; it's
not immediately clear where to draw the line, or even where the fuzzy
middle lies.  The following are some tests that I would use to help make
the decision:

1. Is the functionality so common and well-understood that recent
releases of most dynamic languages all ship with an equivalent module
(or consider it a "standard option", like DBI for Perl 5)?

2. Is there a portable, free library that most existing (non-Parrot)
language implementations all use to provide this functionality?

3. Does the functionality take a great deal of skill/knowledge/effort to
implement?

4. Are minor implementation errors catastrophic (data lost or corrupted,
security broken, etc.)?

5. Is the functionality performance-critical?

6. Is the functionality necessary for accessing a module repository?

7. Is the functionality safe to add without reducing the performance or
increasing the memory usage of Parrot even when that functionality is
*not* in use?

Saying yes to half of those would be a pretty strong indication that the
library deserves to be core, but a module that manages to get 7/7 seems
to me crazy *not* to include.


-'f