[RFC] PDD 16 (NCI) Proposal

Peter Lobsinger plobsing at gmail.com
Thu Mar 18 02:35:43 UTC 2010

On Wed, Mar 17, 2010 at 7:04 PM, Geoffrey Broadwell <geoff at broadwell.org> wrote:
>> * callbacks split into 2 categories: "thunk" and "closure"
> This is a good idea, though the patch feels a little weak in this area.

Yeah, I'll be the first to admit it. As I see it I had 2 problems: the
PDD was too much of a tutorial when it came to callbacks, so I didn't
like the base I was building on; we don't have "closure" callbacks
yet, so I'm guessing at how they'll work. Maybe I need to create some
helper docs to offload some of the explanations and examples from the
PDD so it can run more smoothly.

>> Therefore these signature types are also designed to be passed to a
>> set_string_native vtable on UnManagedStruct to specify the shape of
>> the structure easily. Some additional types may need to be added to
>> support this (eg: explicit padding, smaller integer types for bit
>> packed fields).
> Definitely.  And for UnManagedStruct, some extensions to indicate AoS
> (Array of Structures) layout will be useful as well.  But we can fall
> off that bridge when we come to it.

The bridge is already here. Checkout the implementation of NativeArray
in zavolaj and recoil in appall. The ugly in there is OUR fault.

> OK, on to the patch itself ....
>> +It may not be possible to implement all signature type modifiers. For
>> example,
>> +explicitly sized modifiers are limited by the underlying hardware. In
>> general,
>> +if your C compiler can support it, it is most likely supported by
>> Parrot's NCI.
> We should allow for the possibility that future versions of the NCI
> system (or alternate code generators) will have the ability to support
> sized types that the CPU does not directly handle.  For example, while
> 16-bit floats (so called 'half precision' or 'binary16' floats) have
> been standardized, and are ubiquitous on GPUs, not all CPU families
> support it as a standard native type, and I would guess that many
> compilers have not been updated to support it directly.  Thankfully, the
> 16 <-> 32 bit float conversions are relatively simple and free sample
> code exists, so there's no reason we couldn't support it easily in the
> future.

Point taken. Perhaps a more general statement that we can make no
guarantee (beyond best effort) that all frame builders will support
all signatures on all platforms. What I'm really trying to get at here
is that I'm not going to bend over backwards to implement 128bit
floats on some craptastic i386 (although I'd certainly accept *sane*
patches to accomplish this).

>> +=item 3..7
>> +
>> +Values sized in bits to powers of 2. (eg: 2**4 or 16bits)
> I like this design.  We'll have to have a more exact specification if we
> ever decide to support bitfields (for [Un]ManagedStruct use), but they
> can (and probably should) be handled separately anyway.

Really? I was seriously considering using 'I0", "I1", and "I2" for
bitpacked stuff. I suppose that doesn't allow you to bit pack
full-sized objects that are misaligned, but whoever designs garbage
like that get what they deserve (I've done this). Jury's still out I

>> +=item p, f
>> +C<void *>, and C<void (*)(void)> respectively.
> Hmmmm.  Can we see some examples of exactly how you plan to use these?

I was thinking that parrot already deals with pointers in $IXX
registers (see get_addr).  Sometimes you don't want a new
unmanagedstruct for every return value. For example, if you are
writing a routine that passes around a lot of pointers without
inspecting them, this could be a win. Another scenario I envision is
having 1 unmanaged struct that is used to unpack multiple pointers in
succession by calling set_pointer on it.

The function pointer flag is there because Parrot has gone out of its
way to try and support function pointers being different from data
pointers. Seems like a waste for user code not to be able to handle
the same.

>> +=head4 Float Elements (C<N>)
>> +=item 5..7
>> +
>> +Values sized in bits to powers of 2. (eg: 2**6 = 64bits)
> This should be extended down to 4 (1.5.10 bit fields) at least, if not 3
> (1.3.4 bit fields).
>> +=item f, d, ld
>> +
>> +C<float>, C<double>, and C<long double> respectively. C<ld> must not
>> be
>> +separated by other modifiers.
> May want to add 'h' here as well, for half floats.

Not sure I follow. Are these types only for structs? If they are for
NCI parameters, how do they fit into the ABI if the processor doesn't
support them? But if someone can make it work, the more the merrier.
At least for the sized types. Not sure it makes sense to have a value
that is half the size of the smallest floating point value supported
by the processor.

>> +=head4 String-like Elements (C<S>)
>> +=item p
>> +C<struct { intXX len; char str[] } *>, a counted (Pascal) string. The
>> size of the
>> +C<len> element is determined by size modifiers. The default size is
>> C<sizeof int>.
> Oh, very nice!  We should specify that len is unsigned, I would think.

But then we can't have negative length strings! :p

>> +=item 4..6
>> +Size of the C<len> element of a counted string in bits as a power of
>> two. (eg:
>> +2**6 = 64bits).
> This should be extended down to 3 (8-bit len) -- there's no reason not
> to, and it's in actual use (Forth uses an 8-bit count, for example).

Sure. The cutoff was arbitrary anyways.

>> +=item b
>> +
>> +A fixed-size buffer. When passed by reference, doesn't show up in
>> Parrot return
>> +values. Cannot be a return type.
> I assume you mean 'fixed' as in "caller and callee have arranged the
> length some external way (through an unrelated argument or API
> specification)".  If my assumption is correct, this should be clarified
> in the spec.

This is attempt to emulate the 'b' and 'B' types in the current NCI
implementation. II'm punting here because I'm not exactly sure what
they do. However I am sure I don't like what they seem to do. Maybe I
should drop them. The void ** equivalent for 'B' would simply become
pass by reference string.

>> +Pass by reference types as parameters to callbacks work in a similar
>> fashion.
>> +Their final value is expected as a return value after the C return
>> value.
> This confused me a bit.  Can we see some examples to clarify?

In a callback, if you get passed an int by reference, you're expected
to return an extra value from the callback to set the int.

  # PIR
  .sub parrot_callback
      .param int i
      inc i
      .return (i)

  .sub main :main
     # ...
     $P0 = new_callback parrot_callback, 'Iir->'
     # hand of to C ...

  /* C code */
     int i = 42;
     parrot_callback( &i );
     /* i is 43 here */

>> +=head3 Oversized Values
>> +Parrot will throw an exception if truncation has occurred translating
>> values
>> +from C to Parrot.
> I like the idea of trying and only throwing on truncation, rather than
> just not working period.  What happens in the other direction, though?
> What if a parrot with a 64-bit INTVAL tries to write an I register to a
> 32-bit C function argument and has to truncate?  What about overflowing
> the exponent on a small float type?  Send an infinity or throw an
> exception?

Hmmm... I kind of favour the silent truncation for passing from
parrot. Since parrot is in control of the signatures and the values
passed to these functions, it is Parrot's fault if a value too large
is passed in. It's also very WWCCD (what would cc do?). On the other
hand, there is nothing Parrot can do to handle large values being
passed back. But in situations where Parrot can still cope, it should.
Maybe a warning at either dlfunc or NCI.invoke time about possible
truncation of values.

>> +In the future, wrapping modifiers may be added to the NCI interface
>> to use
>> +C<BigInt>/C<BigNum> PMCs in stead of Parrot-native values.
> This is nice as a future choice, but it leads to a question -- will we
> provide some way for PIR code to detect whether this extension (and
> potentially others, such as fully dynamic signature generation) is
> available?

Does PIR code have a way of knowing about other properties of the
interpreter? Maybe the framebuilder API could be to put a PMC in
interp.iglobals, which would be able to describe it's capabilities.

At the very least, I have resolved to make "signature not supported"
into an exception in stead of a VM panic. Once that's working you can
probe by trying to load functions. If they fail, you know, and you can
presumably deal with it somehow (eg: by loading a dynlib created by
nci_thunk_gen and trying again).

>>  =head3 Callbacks
>> +Unfortunately, much in the same way as call-ins, there's no good way
>> to
>> +generically describe all possible callback parameter sets, so in some
>> cases
>> +hand-written C will be necessary.
> We should provide some guidelines and/or examples of "good practice"
> when doing this.

I don't have much experience with hand-writting C callbacks. Doesn't
opengl do this? Can we just point people at that?

>> +There are 2 posibilities for creating callbacks for C: thunks or
>> closures:
> Thunks are pretty obvious (an improved version of the existing callback
> system).  However, closures are a bit non-obvious in the context of NCI.
> I'd like to see how C closures would actually work, since I'm stuck with
> at least one library that can't use thunks due to lack of userdata
> pointer in callbacks (GLUT), and it would be nice to get rid of the
> bletcherous hack that is libglutcb if I could.

In general C does not support closures. One workaround would be to
compile a bunch of function-static var pairs and hope you don't
exhaust your supply. Another is to drop down into assembly. libffi
provides a closure interface on *some* platforms. Given enough
motivation, people could add support for their platform using either
libffi as a base or doing the asm themselves. That is, unless the
platform has mechanisms that make native closures impossible, in which
case we're SOL, but there's nothing you can do anyways.

More information about the parrot-dev mailing list