"Blocking Buffered Stream" threading primitive

[Daniel Ruoso]

2011/12/26 Nicholas Clark <nick at ccl4.org>:
> On Sun, Dec 25, 2011 at 03:31:36PM -0500, Daniel Ruoso wrote:
>> The point I'm trying to address is the one that in Perl 6 we are going
>> to have a lot of implicit threading constructs, such as:
>> my @a <== grep { foo() } <== map { bar() } <== 1..inf;
> Does Perl 6 mandate a threading model akin to POSIX, were the threads of
> execution all run with shared state, something more like Perl 5 ithreads
> (or I guess a message passing system) where everything is decoupled, or
> "it doesn't mandate anything"?

It doesn't mandate anything, but I think the general expectation is
that you will be able to read and write to values accessible to the
grep and map closures. On the other hand, write operations can be
pessimized by using a message-passing proxy that will talk with a task
running in the OS thread where the external object lives. If you just
use the input in map and grep it will work completely parallel, but if
you keep writing to outside values, it will be implicitly synchronized
(but just the access to that value, but the order in which this access
happen is still undefined).

>> What do you think?
> Presumably the Perl 6 compiler is only allowed to implicitly parallelise the
> above construction if it can be sure that foo() and bar() have no side
> effects?

That is not the case, map and grep will not run in order and it is in
general accepted that such constructs will run asynchronously. So,
while it is not forbidden to have side effects, the user will know
that the result is not guaranteed to have any ordering consistency.

> Presumably this sort of threading only pays off if the CPU gains from
> using multiple cores more than offset the cost of creating the threads, and
> passing return results back?

I think the parameters on how the tasks will be spread on different OS
threads is something to be fine-tuned (ghc does that, for instance).
At this moment, the important thing is to properly declare that as
concurrent tasks (which is what the feed operator does) and have a way
to build the flow of data controlling when to block/unblock each task.

That is why I'm considering the idea of a "blocking buffered stream"
VM primitive (and why this thread is on this list), since basically
this will provide a non-OS way to implement blocking reads and writes,
which will effect how the scheduler chooses which task to run. As I
said in the original post, I think the only way to make it efficient
is doing it in the VM level.

daniel

>> P.S.: I am not subscribed to the parrot-dev list, please CC-me.
P.S.: I am actually subscribed now, so you don't need to CC-me anymore