Assorted C-FFI goodies 
Author Message
 Assorted C-FFI goodies

Thanks to everyone who helped me with the macro questions! I managed
to get the c-function-definer macros to work after about a half hour
of hacking, and I've gotten the C-FFI library to parse; I'm now
working on compilation. :)

I've thoroughly beaten this to death on gd-hackers, but now that I
understand more of the structure of the C-FFI IDL, I'd like to
resurrect some of this here to get a wider audience. :)

I'm seeing the following in c-ffi.dylan:

/* XXX - Need clear-memory!, copy-bytes!, copy-into!, equal-memory? */

Where can I find prototypes for these?

Also, I've never worked on code to communicate between languages
before, so I'm curious about something. How, exactly, does one coerce
C primitives into Dylan objects? I guess I could see how it is
possible with pointers; just copy the bytes using the above function
and structure them into an integer, right? Or, is there more than
that? How does one do that with values as opposed to references,
though? Is 'define C-subtype' the macro which I'd use for this? Or are
the primitives handled in a different manner?

Also, how would function calls work?

And, having admitted that I've never done something like this before,
I'm curious -- is this something that I could probably finish, or does
a project like this require lots of knowledge about the inner workings
of C primitives and such? I'd very much like to finish this since I'm
extremely interested in using Dylan for regular projects, but if this
is more than likely beyond my abilities, then I should probably shift
my efforts to Pidgin and wait for someone else to take up
c-ffi. Thoughts?



Thu, 24 Oct 2002 03:00:00 GMT  
 Assorted C-FFI goodies

    Chris> In the Functional Developer help, C-FFI and Win32
    Chris> Reference, section 1.9, Utility Designator classes:

    Chris> clear-memory!(pointer, size) => ()
    Chris> copy-bytes!(destination-pointer, source-pointer, size) =>
    Chris> () copy-into!(destination-pointer, source-pointer, size) =>
    Chris> () equal-memory?(ptr1, ptr2, size) => <boolean>

Interesting. The documentation refers to <pointer> types, but all I
can see in GD is a <raw-pointer>:

define functional class <raw-pointer> (<object>)
  sealed slot value :: <raw-pointer>, required-init-keyword: value:;
end;

Is this the same thing? Or, is it actually refering to a <C-pointer>?



Thu, 24 Oct 2002 03:00:00 GMT  
 Assorted C-FFI goodies

Quote:

> Also, I've never worked on code to communicate between languages
> before, so I'm curious about something. How, exactly, does one coerce
> C primitives into Dylan objects? I guess I could see how it is
> possible with pointers; just copy the bytes using the above function
> and structure them into an integer, right? Or, is there more than
> that? How does one do that with values as opposed to references,
> though? Is 'define C-subtype' the macro which I'd use for this? Or are
> the primitives handled in a different manner?

Non-pointer value types are converted to and from Dylan value types
(either <integer> or <machine-word>; see table 1.1 in the C-FFI
documentation) whenever they are imported or exported.

For object types, the C-subtype-definer macro allows Dylan types to be
defined that serve as designator classes, proxies for corresponding C
types.  Designator class objects are like <class> objects, except that
they contain some additional information (import type, export type,
import function, export function, pointer type, and referenced type).

In fact, the C-struct-definer, C-union-definer, C-subtype-definer,
C-mapped-subtype-definer, and C-pointer-type-definer macros can all be
defined in terms of a hook into the compiler that might be called
designator-class-definer.  This would define a new designator
metaclass object, giving the compiler enough information to do it's
job.

Quote:
> Also, how would function calls work?

C function calls can be implemented using the call-out primitive,
exported from the System module of the Dylan library.  This can be
used to call specific functions, or to call through function pointers.

Quote:
> And, having admitted that I've never done something like this before,
> I'm curious -- is this something that I could probably finish, or does
> a project like this require lots of knowledge about the inner workings
> of C primitives and such? I'd very much like to finish this since I'm
> extremely interested in using Dylan for regular projects, but if this
> is more than likely beyond my abilities, then I should probably shift
> my efforts to Pidgin and wait for someone else to take up
> c-ffi. Thoughts?

The C-FFI library itself is just the tip of the iceberg, an extension
of the real mechanism that needs to be buried inside the compiler.
Once the compiler machinery is in place the rest of the library should
fall into place pretty quickly.

Unless you've experienced working with the innards of compilers I
would recommend not opening this particular can of worms.  Pidgin
would be a much easier project to deal with, especially since you can
use Functional Developer to test it on.

I started working on building the compiler infrastructure required for
supporting C-FFI while vacationing at my in-laws' in Taiwan last
April.  Since I was using a laptop with a tiny screen and 64M of RAM
(not enough to recompile d2c in a timely manner) in two weeks of work
I only managed to push the requisite informaton about one third of the
way through the compiler.

I think this compiler (d2c) is a bit too complex for its own good.  I
have hopes for designing a more elegant one in connection with my
graduate research in the next couple of years.  Before that happens,
though, I've decided to go ahead and finish the
<designator-class>/C-FFI work.  So don't sweat C-FFI for now.

By the way, I'll start a CVS branch and put what I've done so far in
it, but don't expect it to be able to compile it for awhile yet.

Cheers,



Thu, 24 Oct 2002 03:00:00 GMT  
 Assorted C-FFI goodies

Quote:

> /* XXX - Need clear-memory!, copy-bytes!, copy-into!, equal-memory? */

> Where can I find prototypes for these?

In the Functional Developer help, C-FFI and Win32 Reference, section
1.9, Utility Designator classes:

clear-memory!(pointer, size) => ()
copy-bytes!(destination-pointer, source-pointer, size) => ()
copy-into!(destination-pointer, source-pointer, size) => ()
equal-memory?(ptr1, ptr2, size) => <boolean>

Quote:

> How, exactly, does one coerce C primitives into Dylan objects?

It might help to have a look at the melange sources, and the other
aspects of GD that does this sort of thing. Is it possible to use the
c-ffi stuff you are doing to be converted into the current means of
working with C with GD (Melange, etc). This might involve less work.

Chris.
--
http://www.double.co.nz/dylan



Fri, 25 Oct 2002 03:00:00 GMT  
 Assorted C-FFI goodies

    Peter> The C-FFI library itself is just the tip of the iceberg, an
    Peter> extension of the real mechanism that needs to be buried
    Peter> inside the compiler.  Once the compiler machinery is in
    Peter> place the rest of the library should fall into place pretty
    Peter> quickly.

    Peter> Unless you've experienced working with the innards of
    Peter> compilers I would recommend not opening this particular can
    Peter> of worms.  Pidgin would be a much easier project to deal
    Peter> with, especially since you can use Functional Developer to
    Peter> test it on.

Hmm, now I'm curious. :)

I've been toying with call-out now that I've discovered it. What
components of c-ffi need to be buried in the compiler?

In studying the specification a bit more closely, I began to determine
that I'm not interested in a full and complete c-ffi implementation at
the moment. Unless I'm mistaken, the complete definition allows for
Dylan functions to be called from C, which I'm not particularly
interested in at the moment. If I sacrifice that functionality, then
how difficult would it be to achieve access to C data from within
Dylan? With call-out, is it possible to implement a Dylan function
which simply runs call-out on the results of a macro call and returns
the result variable? What about access to C variables? Would the
c-decl macros within the System module be useful for this? Or am I
missing something important?

    Peter> I started working on building the compiler infrastructure
    Peter> required for supporting C-FFI while vacationing at my
    Peter> in-laws' in Taiwan last April.  Since I was using a laptop
    Peter> with a tiny screen and 64M of RAM (not enough to recompile
    Peter> d2c in a timely manner) in two weeks of work I only managed
    Peter> to push the requisite informaton about one third of the way
    Peter> through the compiler.

Ah, heh. I just grabbed another 128 megs of RAM; the 64 meg situation
was killing me until last Friday. Hence my sudden drastic increase in
activity and interest. :)

    Peter> I think this compiler (d2c) is a bit too complex for its
    Peter> own good.  I have hopes for designing a more elegant one in
    Peter> connection with my graduate research in the next couple of
    Peter> years.  Before that happens, though, I've decided to go
    Peter> ahead and finish the <designator-class>/C-FFI work.  So
    Peter> don't sweat C-FFI for now.

    Peter> By the way, I'll start a CVS branch and put what I've done
    Peter> so far in it, but don't expect it to be able to compile it
    Peter> for awhile yet.

That'd be awesome! If this is as difficult as it is then I'm not sure
if I can help, but I'd be interested in seeing if I'm on the right
track with this. All I've had to go on for the past few days is the
warning/error ratio which I'm slowly decreasing, but I could be
completely botching everything. I tried writing my first memory
address and bit manipulation functions today, and probably did them
completely wrong considering that everything I learned about those two
subjects was gleamed from a casual glance over the d2c/runtime/dylan
sources. :)



Fri, 25 Oct 2002 03:00:00 GMT  
 Assorted C-FFI goodies

Quote:

> I've been toying with call-out now that I've discovered it.

call-out is cool. I've hand-written my Mac Toolbox library by hand using
call-out. It's very, very tedious but not difficult to do.
c-expr is good, too. I use that a little but Gabor Greif posted a m,uch
better example of (mis-)using it on dylan-hackers a while back.
It might be worthwhile getting Pidgin to spew call-out / c-expr calls and
<machine-pointer> types rather than write to the C-FFI, which isn't
complete yet by a long shot.

Quote:
> What components of c-ffi need to be buried in the compiler?

Type mapping and translation, call translation, callback translation.
Stack frames don't build themselves. :-)

Quote:
> Unless I'm mistaken, the complete definition allows for
> Dylan functions to be called from C, which I'm not particularly
> interested in at the moment.

Melange and the current call-out style FFI allow callbacks. They are
important for interfacing to Operating Systems. MacOS needs them and I'm
sure BeOS does, too. X is entirely callback based, if I remember
correctly.

Quote:
> If I sacrifice that functionality, then
> how difficult would it be to achieve access to C data from within
> Dylan?

Use c-expr, or <statically-typed-pointer> and the (int|long|char)-at
methods and you can do that today. See the Mac toolbox lib for examples
(they're hand written, so they have to be human readable). Gareth Baker's
Carbon application example has melange-parsed interfaces to the same
functionality, which make a good comparison. It also shows how callbacks
are created.

Quote:
> With call-out, is it possible to implement a Dylan function
> which simply runs call-out on the results of a macro call and returns
> the result variable?

Absolutely. I don't know about type safety, though.

define macro dummy
    { dummy( ?val:expression ) }
        =>( call-out( "GetValue", int:, int: unseen-macro( as(
<c-string>, ?val ).raw-value  ) )  )
end macro dummy;

Quote:
> What about access to C variables? Would the
> c-decl macros within the System module be useful for this?

Yes. The d2c sources import errno from the std c libs in this way. See
Gabor's example from the gwydion list, which creates, manipulates and
passes C variables from within Dylan (this may be applicable to the
poster asking about embedding C in Dylan, too).

Quote:
> Or am I missing something important?

No.

Quote:
> That'd be awesome! If this is as difficult as it is then I'm not sure
> if I can help, but I'd be interested in seeing if I'm on the right
> track with this.

Any and all help on d2c is greatly appreciated, and people should play to
their strengths. I'm working on DUIM because I don't know how to finish
the C-FFI or class/each-subclass slots no matter how badly they're
needed.

Please take a look at the Mac Toolbox examples from mac-d2c and
carbon-application, I'm sure they'll help with understanding how call-out
and friends work. I'm happy to answer questions on it.

- Rob.



Fri, 25 Oct 2002 03:00:00 GMT  
 Assorted C-FFI goodies

Quote:

> Ah, heh. I just grabbed another 128 megs of RAM; the 64 meg situation
> was killing me until last Friday. Hence my sudden drastic increase in
> activity and interest. :)

When I started playing with d2c I had 32 MB in my Linux machine -- which had
been more than adequate up to that point.  Builds took 24 hours, with under
5% CPU utilization.  Adding 128 MB of RAM fixed that :-)

-- Bruce



Fri, 25 Oct 2002 03:00:00 GMT  
 Assorted C-FFI goodies

    Rob> call-out is cool. I've hand-written my Mac Toolbox library by
    Rob> hand using call-out. It's very, very tedious but not
    Rob> difficult to do.  c-expr is good, too. I use that a little
    Rob> but Gabor Greif posted a m,uch better example of (mis-)using
    Rob> it on dylan-hackers a while back.  It might be worthwhile
    Rob> getting Pidgin to spew call-out / c-expr calls and
    Rob> <machine-pointer> types rather than write to the C-FFI, which
    Rob> isn't complete yet by a long shot.

Hmm. I always thought that call-out and friends were Melange-isms, but
now that I know they aren't ... :)

This is a silly question, but where exactly is the toolbox lib? Is it
./mindy/Macintosh/toolbox-lib/? If so, I don't see call-outs, though I
do see a get-c-function call used here, but that seems to be a
Mindyism. Also, what is the point of c-include? Does that include the
header in the outpputted C code?



Fri, 25 Oct 2002 03:00:00 GMT  
 Assorted C-FFI goodies

Quote:

> Subject: Re: Assorted C-FFI goodies

> This is a silly question, but where exactly is the toolbox lib? Is it
> ./mindy/Macintosh/toolbox-lib/?

You need gd/macintosh/mac-d2c2.3.3r2.sit.hqx (or nearest correct spelling).

Quote:
> Also, what is the point of c-include? Does that include the
> header in the outpputted C code?

Yes, it inserts an #include.

- Rob.



Fri, 25 Oct 2002 03:00:00 GMT  
 Assorted C-FFI goodies

    Rob> You need gd/macintosh/mac-d2c2.3.3r2.sit.hqx (or nearest
    Rob> correct spelling).

Hrm. Any ideas about how I can get at this from Linux? I tried
uudeview on the file and got the .sit, but I can't figure out how to
extract the stuffit archive and get the files. :)



Fri, 25 Oct 2002 03:00:00 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. to CS: or not to CS: in F-PC assembler

2. FA assorted programming reference books

3. Assorted simple Standard ML modules

4. assorted WEL questions (WEL_TREE_VIEW, OPEN_FILE_DIALOG)

5. Log4r 1.0.1 - remote logging and assorted updates

6. Looking for Assorted Ada Tools

7. Assorted Questions regarding the future of Python...

8. Assorted Powerstation problems

9. Comparing TCL on Cygwin and on Scriptics assorted versions after 7.0

10. Tcl/Tk, Unix/Win/MacOS, CW Pro1 -- assorted questions (LONG)

11. Newbie..assorted

 

 
Powered by phpBB® Forum Software