[RFC] PDD 16 (NCI) Proposal

Geoffrey Broadwell geoff at broadwell.org
Thu Mar 18 04:03:30 UTC 2010

On Wed, 2010-03-17 at 22:35 -0400, Peter Lobsinger wrote:
> Yeah, I'll be the first to admit it. As I see it I had 2 problems: the
> PDD was too much of a tutorial when it came to callbacks, so I didn't
> like the base I was building on;


> we don't have "closure" callbacks
> yet, so I'm guessing at how they'll work.

Sure.  In that case, we should mark these as [Conjecture] as Perl 6
specs often do when there is (more than usual) expectation that actual
implementation experience will change the design.

> Maybe I need to create some
> helper docs to offload some of the explanations and examples from the
> PDD so it can run more smoothly.

Yes, NCI needs tutorial docs.  Not just tests (which last I saw were way
insufficient), and not just random examples, but 'parrotncitut'.  For
now, it's probably a fair start to just break the existing stuff into a
separate doc and point to it.

> > Definitely.  And for UnManagedStruct, some extensions to indicate AoS
> > (Array of Structures) layout will be useful as well.  But we can fall
> > off that bridge when we come to it.
> The bridge is already here. Checkout the implementation of NativeArray
> in zavolaj and recoil in appall. The ugly in there is OUR fault.

Point taken.  Well, let's get this patch cleaned up and committed, and
start working on the next patch ([Un]ManagedStruct, AoS, etc.) after
that.  If we try to do it all at once we risk getting bogged down, and
it's not like we have to pull this PDD out of draft tomorrow.  :-)

> Point taken. Perhaps a more general statement that we can make no
> guarantee (beyond best effort) that all frame builders will support
> all signatures on all platforms. What I'm really trying to get at here
> is that I'm not going to bend over backwards to implement 128bit
> floats on some craptastic i386 (although I'd certainly accept *sane*
> patches to accomplish this).


> Really? I was seriously considering using 'I0", "I1", and "I2" for
> bitpacked stuff. I suppose that doesn't allow you to bit pack
> full-sized objects that are misaligned, but whoever designs garbage
> like that get what they deserve (I've done this). Jury's still out I
> guess.

A relatively common packing trick for structures with both one or more
pointer fields and some boolean flags is to align the structure to a
2^(num_flags/num_pointers) byte boundary and use the bottom bits of the
pointers to store the flags.  One of the libraries I used in the past
had split the pointers 29/3 or 30/2, I forget.

Anyway, that requires something different than I0 .. I2, because the
fields aren't all 2^n bits wide.

> I was thinking that parrot already deals with pointers in $IXX
> registers (see get_addr).  Sometimes you don't want a new
> unmanagedstruct for every return value. For example, if you are
> writing a routine that passes around a lot of pointers without
> inspecting them, this could be a win. Another scenario I envision is
> having 1 unmanaged struct that is used to unpack multiple pointers in
> succession by calling set_pointer on it.
> The function pointer flag is there because Parrot has gone out of its
> way to try and support function pointers being different from data
> pointers. Seems like a waste for user code not to be able to handle
> the same.

OK, fair enough.

> > [small floats]
> Not sure I follow. Are these types only for structs? If they are for
> NCI parameters, how do they fit into the ABI if the processor doesn't
> support them? But if someone can make it work, the more the merrier.
> At least for the sized types. Not sure it makes sense to have a value
> that is half the size of the smallest floating point value supported
> by the processor.

Well ... some systems deal with these by unioning with a short int;
others only handle them as part of packed vector data (so the normal
C-level code can't handle it, but the assembly intrinsics can).

> > I assume you mean 'fixed' as in "caller and callee have arranged the
> > length some external way (through an unrelated argument or API
> > specification)".  If my assumption is correct, this should be clarified
> > in the spec.
> This is attempt to emulate the 'b' and 'B' types in the current NCI
> implementation. II'm punting here because I'm not exactly sure what
> they do. However I am sure I don't like what they seem to do. Maybe I
> should drop them. The void ** equivalent for 'B' would simply become
> pass by reference string.

Gotcha.  I don't want to just drop this functionality, but I agree we
need to find a better/clearer way to deal with it.

> > This confused me a bit.  Can we see some examples to clarify?
> In a callback, if you get passed an int by reference, you're expected
> to return an extra value from the callback to set the int.
>   # PIR
>   .sub parrot_callback
>       .param int i
>       inc i
>       .return (i)
>   .end
>   .sub main :main
>      # ...
>      $P0 = new_callback parrot_callback, 'Iir->'
>      # hand of to C ...
>   .end
>   /* C code */
>   {
>      int i = 42;
>      parrot_callback( &i );
>      /* i is 43 here */
>    }

Oh, I understand perfectly now.  That should go with your other
examples, in the sample/tutorial doc.  :-)

> Hmmm... I kind of favour the silent truncation for passing from
> parrot. Since parrot is in control of the signatures and the values
> passed to these functions, it is Parrot's fault if a value too large
> is passed in. It's also very WWCCD (what would cc do?). On the other
> hand, there is nothing Parrot can do to handle large values being
> passed back. But in situations where Parrot can still cope, it should.

Agreed with all this.

> Maybe a warning at either dlfunc or NCI.invoke time about possible
> truncation of values.

Hmmmm.  I'm not sure I like the warning.  Let's try just clearly
speccing silent truncation, and see who screams.

> Does PIR code have a way of knowing about other properties of the
> interpreter? Maybe the framebuilder API could be to put a PMC in
> interp.iglobals, which would be able to describe it's capabilities.

Actually, the number of ways to get info about the interpreter bugs me,
and I've never really understood *why* this was not unified.  Just count
the different ways I have to gather data in the INIT block of Plumage's
Util library:


> At the very least, I have resolved to make "signature not supported"
> into an exception in stead of a VM panic.


> Once that's working you can
> probe by trying to load functions. If they fail, you know, and you can
> presumably deal with it somehow (eg: by loading a dynlib created by
> nci_thunk_gen and trying again).

Makes sense.  That doesn't handle such questions as whether large
numbers will unpack into a BigInt or throw an exception on overflow, but
it's a start.

> > We should provide some guidelines and/or examples of "good practice"
> > when doing this.
> I don't have much experience with hand-writting C callbacks. Doesn't
> opengl do this? Can we just point people at that?

Right now, there's just the ugliness that is libglutcb for GLUT
callbacks, and I wouldn't call it a shining example of best practice (I
tried to make it as clean as I could with the tools I had available, but
that's not saying much).  I haven't tackled the GLU callbacks, which are
a *mix* of callbacks with and without a user data parameter.  Ewww.

> In general C does not support closures.


> One workaround would be to
> compile a bunch of function-static var pairs and hope you don't
> exhaust your supply.

Fragile, but so is the libglutcb method.

> Another is to drop down into assembly. libffi
> provides a closure interface on *some* platforms. Given enough
> motivation, people could add support for their platform using either
> libffi as a base or doing the asm themselves. That is, unless the
> platform has mechanisms that make native closures impossible, in which
> case we're SOL, but there's nothing you can do anyways.

Fair enough.

OK, want to spin a new patch for further comment?


More information about the parrot-dev mailing list