green_threads

[Stefan Seifert]

Dear parrot developers,

during the past two months I've been working on bringing the gsoc_threads 
branch up to date and into a usable shape. Now I got to the point were serious 
talk about merging and future development is due. fulltest passes, as does 
Rakudo's spectests. On git at github.com:niner/parrot.git I have a green_threads 
branch containing my work.

So what are "green threads"? Green threads or tasks are lightweight threads. 
They exist only within the interpreter and do not use operating system 
facilities like threads or processes. Instead, the interpreter itself contains 
a scheduler which manages running tasks and switches between them.

Where are we now? Parrot_pf_execute_bytecode_program got changed to instead of 
calling the main sub directly, starting up the scheduler and running the main 
sub as the first task. The scheduler preemptively switches tasks every 20ms. 
Tasks can be started directly or as result of an expired alarm or a fireing 
timer. Attached is a little test program demonstrating task switching.

There are some restrictions to the current implementation:
* Blocking I/O and native calls block the whole interpreter, so no task
  switches occur
* Preemptive task switching only works if the current task is in the top most
  runloop. Otherwise the task switch is skipped and hopefully the task
  returned to runloop 1 for the next task switch
* Voluntary task switching (calling sleep or pass) suffers the same
  restriction. Even worse, trying it anyway will currently end up in a corrupt
  stack. A workaround will be to block the whole interpreter for sleep and
  probably ignore a pass.

In short: runloops are a real hassle. But tasks can still be useful. One could 
use them for handling signals for example.

Possible ways forward from here:
An immediate improvement will be to stop preemption if only one task is 
active. Or the other way around: start the scheduler's timer only when a 
second task gets added to the list. This way, tasks will have zero overhead in 
the common case.

The restriction of blocking I/O could be circumvented by using asynchronous 
I/O instead and and suspending the task in the meanwhile. This would be a nice 
performance improvement for I/O bound programs.

Obviously real threads could fix this as well. They seem inevitable in the long 
run anyway. But of course, those are a huge amount of work.

Any change that removes nested runloops will be very much welcomed :)

In any case, I think it's time to start some serious communications with 
Rakudo folks about these features and where to go from now. Most of all we 
need feedback.

Stefan
-------------- next part --------------
#!./parrot

.sub main :main
    .local pmc task, a, b
    task = get_global 'task'
    a = new ['String']
    a = "a"
    b = new ['String']
    b = "b"
    $P0 = new ['Task']
    setattribute $P0, 'code', task
    setattribute $P0, 'data', a
    $P1 = new ['Task']
    setattribute $P1, 'code', task
    setattribute $P1, 'data', b
    schedule $P0
    schedule $P1
.end

.sub task
    .param pmc name
    .local int i
start:
    print name
    i = 0
loop:
    inc i
    if i >= 100000 goto start
    goto loop
.end
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.parrot.org/pipermail/parrot-dev/attachments/20111011/abcf14cf/attachment.asc>