[RFC] PDD 16 (NCI) Proposal

Geoffrey Broadwell geoff at broadwell.org
Wed Mar 17 23:04:39 UTC 2010


On Tue, 2010-03-16 at 23:13 -0400, Peter Lobsinger wrote:
> Please find attached a patch for PDD 16 that addresses many of the
> concerns I am aware of regarding NCI.

Thank you for working on this!  First, some comments on the summary:

> Notable changes include:
> * a move to PCC-alike signatures
> * support for a wider range of types (unsigned, pass by reference,
> large types, etc)

Yay!

> * '2', '3', and '4' PMC integer pass by reference types have been
> replaced with a more general pass by reference system

Much more reasonable.  This always bothered me.

> * explicit flagging and handling of memory ownership of string buffers

Good idea.

> * callbacks split into 2 categories: "thunk" and "closure"

This is a good idea, though the patch feels a little weak in this area.

> * thunk callbacks no longer modify user supplied userdata parameter
> * memory management of callback state explicitly delegated to user

Good!

> (we currently leak memory to avoid this)

>_<

> * brief documentation of the shortcomings of the default (static)
> frame builder, mention of the possibility of others

Good idea.

> Some concerns *not* addressed by this (but mentioned in NCITasklist) are:
> * wide characters
> * padded strings/buffers
> * zero-copy buffers

We'll get there.  What you have written is a good start.

> It is my opinion that these are the realm of the string system and/or
> custom PMCs (even though they may be very useful for interacting
> *with* NCI).

I think the proper path here will probably need to be determined by
real-world use.

> Note that some of the proposed NCI parameter types may be of dubious
> value for only NCI (especially 'Iv" and "In"). However, it is my
> intention to make the use of NCI and UnManagedStruct consistent.

And there was much rejoicing ....  (I particularly wanted byte ordering
support, so this made me smile.)

> Therefore these signature types are also designed to be passed to a
> set_string_native vtable on UnManagedStruct to specify the shape of
> the structure easily. Some additional types may need to be added to
> support this (eg: explicit padding, smaller integer types for bit
> packed fields).

Definitely.  And for UnManagedStruct, some extensions to indicate AoS
(Array of Structures) layout will be useful as well.  But we can fall
off that bridge when we come to it.

OK, on to the patch itself ....

> +It may not be possible to implement all signature type modifiers. For
> example,
> +explicitly sized modifiers are limited by the underlying hardware. In
> general,
> +if your C compiler can support it, it is most likely supported by
> Parrot's NCI.

We should allow for the possibility that future versions of the NCI
system (or alternate code generators) will have the ability to support
sized types that the CPU does not directly handle.  For example, while
16-bit floats (so called 'half precision' or 'binary16' floats) have
been standardized, and are ubiquitous on GPUs, not all CPU families
support it as a standard native type, and I would guess that many
compilers have not been updated to support it directly.  Thankfully, the
16 <-> 32 bit float conversions are relatively simple and free sample
code exists, so there's no reason we couldn't support it easily in the
future.

> +=item 3..7
> +
> +Values sized in bits to powers of 2. (eg: 2**4 or 16bits)

I like this design.  We'll have to have a more exact specification if we
ever decide to support bitfields (for [Un]ManagedStruct use), but they
can (and probably should) be handled separately anyway.

> +=item s, i, l, ll
>  
> +C<short>, C<int>, C<long>, C<long long> respectively. C<ll> must not
> be
> +separated by other modifiers or it will be parsed as two C<l> size
> modifiers.

Excellent, thank you.

> +=item p, f
>  
> +C<void *>, and C<void (*)(void)> respectively.

Hmmmm.  Can we see some examples of exactly how you plan to use these?

> +=head4 Float Elements (C<N>)
> +=item 5..7
> +
> +Values sized in bits to powers of 2. (eg: 2**6 = 64bits)

This should be extended down to 4 (1.5.10 bit fields) at least, if not 3
(1.3.4 bit fields).

> +=item f, d, ld
> +
> +C<float>, C<double>, and C<long double> respectively. C<ld> must not
> be
> +separated by other modifiers.

May want to add 'h' here as well, for half floats.

> +=head4 String-like Elements (C<S>)
> +=item p
>  
> +C<struct { intXX len; char str[] } *>, a counted (Pascal) string. The
> size of the
> +C<len> element is determined by size modifiers. The default size is
> C<sizeof int>.

Oh, very nice!  We should specify that len is unsigned, I would think.

> +=item 4..6
>  
> +Size of the C<len> element of a counted string in bits as a power of
> two. (eg:
> +2**6 = 64bits).

This should be extended down to 3 (8-bit len) -- there's no reason not
to, and it's in actual use (Forth uses an 8-bit count, for example).

> +=item b
> +
> +A fixed-size buffer. When passed by reference, doesn't show up in
> Parrot return
> +values. Cannot be a return type.

I assume you mean 'fixed' as in "caller and callee have arranged the
length some external way (through an unrelated argument or API
specification)".  If my assumption is correct, this should be clarified
in the spec.

> +=head3 Pass By Reference
> [...]
> +the value of the argument after the function call will be used as a
> return
> +value.
> +
> +Pass by reference return values are placed after the C return value
> and occur
> +in the same order as they are encountered in the arguments signature.

All good.

> +Pass by reference types as parameters to callbacks work in a similar
> fashion.
> +Their final value is expected as a return value after the C return
> value.

This confused me a bit.  Can we see some examples to clarify?

> +=head3 Oversized Values
> +Parrot will throw an exception if truncation has occurred translating
> values
> +from C to Parrot.

I like the idea of trying and only throwing on truncation, rather than
just not working period.  What happens in the other direction, though?
What if a parrot with a 64-bit INTVAL tries to write an I register to a
32-bit C function argument and has to truncate?  What about overflowing
the exponent on a small float type?  Send an infinity or throw an
exception?

> +In the future, wrapping modifiers may be added to the NCI interface
> to use
> +C<BigInt>/C<BigNum> PMCs in stead of Parrot-native values.

This is nice as a future choice, but it leads to a question -- will we
provide some way for PIR code to detect whether this extension (and
potentially others, such as fully dynamic signature generation) is
available?

>  =head3 Callbacks
> +Unfortunately, much in the same way as call-ins, there's no good way
> to
> +generically describe all possible callback parameter sets, so in some
> cases
> +hand-written C will be necessary.

We should provide some guidelines and/or examples of "good practice"
when doing this.

> +There are 2 posibilities for creating callbacks for C: thunks or
> closures:

Thunks are pretty obvious (an improved version of the existing callback
system).  However, closures are a bit non-obvious in the context of NCI.
I'd like to see how C closures would actually work, since I'm stuck with
at least one library that can't use thunks due to lack of userdata
pointer in callbacks (GLUT), and it would be nice to get rid of the
bletcherous hack that is libglutcb if I could.

All in all, good work, thanks again!


-'f




More information about the parrot-dev mailing list