using test::unit for C++ unit tests 
Author Message
 using test::unit for C++ unit tests

I'm going to be starting on a C++ development project for a contract I'm
doing and I'd like to use the 'test-first' philosophy and develop unit
tests first.  Has anyone adapted any of Ruby's unit testing frameworks to
do unit testing for C++ classes?

I'm thinking that there could be a couple of approaches:
1) make the C++ libraries Ruby extensions and instaniate the C++ objects
and pass messages to them on the Ruby side.
2) make some kind of wrapper class (in C++) that gets used to instantiate
and call methods and then returns some error code.  Compile this wrapper
into an executable binary that gets called from Ruby with error codes
being passed back:
errors = 'exercise <some params>'

3) a variation on #2 that uses expect.rb.

While #1 seems perhaps to be the cleanest choice, we don't have much time
to set all of this up, so creating the extensions might take longer than
what we'd like.

Can anyone who has done this offer some pointers?

Phil



Fri, 12 Nov 2004 04:26:04 GMT  
 using test::unit for C++ unit tests
Phil:

As a C/C++ guy at heart, I have given some thought to this problem (i.e.,
that of using Ruby to generate unit tests for other languages like, oh, say
C and C++). In fact, it's one of the core themes of the book I'm working on
with Rich Kilmer and Dana Moore: Ruby as a software engineering toolsmith
language.

One approach (the one toward which I lean) is to generate the C++ classes
using Ruby as a sort of code specification language. I'm calling the package
RuGen, and it will have an assortment of code generators one day, but
upfront I'm working on code generators for Java and C++. The beauty of this
approach is it becomes unnecessary to parse a C++ file to determine what to
test, because you already have all of the metadata about the code you need
to a.) generate the code (or at least the complete skeleton/interface
thereof) and b.) generate the test code for the generated code, treating the
interface as a "black box". (I'm writing a separate package, RuLog, to
provide the logic/inference capabilities necessary to create meaningful and
comprehensive test cases from the spec).

RuGen is still in the design/prototype phase, but the idea is fairly clear
in my head, in terms of how it will work. Rather than create a bunch of
specific language parsers, have one specification language (in much the same
way that Ruby/DL is a specification language for dynamic loading of external
libraries) and a number of generators that are capable of translating the
spec to code in a more or less plug'n'play, or, if I ever get around to
slapping a GUI on it, a point'n'click fashion.

Hopefully I'll have something working soon - I'm probably going to do the
Java generator first, though. But it should serve as an example to everybody
else how to build a code generator for the RuGen package (RuGen providing
the high-level abstractions needed to tie all the plug-in pieces together).

All thoughts, comments, suggestions welcome.

:)

Bob Calco

Quote:
% -----Original Message-----


% Sent: Sunday, May 26, 2002 2:00 PM
% To: ruby-talk ML
% Subject: using test::unit for C++ unit tests
%
%
% I'm going to be starting on a C++ development project for a contract I'm
% doing and I'd like to use the 'test-first' philosophy and develop unit
% tests first.  Has anyone adapted any of Ruby's unit testing frameworks to
% do unit testing for C++ classes?
%
% I'm thinking that there could be a couple of approaches:
% 1) make the C++ libraries Ruby extensions and instaniate the C++ objects
% and pass messages to them on the Ruby side.
% 2) make some kind of wrapper class (in C++) that gets used to instantiate
% and call methods and then returns some error code.  Compile this wrapper
% into an executable binary that gets called from Ruby with error codes
% being passed back:
% errors = 'exercise <some params>'
%
% 3) a variation on #2 that uses expect.rb.
%
%
% While #1 seems perhaps to be the cleanest choice, we don't have much time
% to set all of this up, so creating the extensions might take longer than
% what we'd like.
%
% Can anyone who has done this offer some pointers?
%
% Phil
%



Fri, 12 Nov 2004 08:55:39 GMT  
 using test::unit for C++ unit tests

Quote:
> I'm going to be starting on a C++ development project for a contract I'm
> doing and I'd like to use the 'test-first' philosophy and develop unit
> tests first.  Has anyone adapted any of Ruby's unit testing frameworks to
> do unit testing for C++ classes?

How about CppUnit?

  http://cppunit.sourceforge.net/

I hear CppUnit is miserable, though.

Quote:
> I'm thinking that there could be a couple of approaches:
> 1) make the C++ libraries Ruby extensions and instaniate the C++ objects
> and pass messages to them on the Ruby side.
> 2) make some kind of wrapper class (in C++) that gets used to instantiate
> and call methods and then returns some error code.  Compile this wrapper
> into an executable binary that gets called from Ruby with error codes
> being passed back:
> errors = 'exercise <some params>'

> 3) a variation on #2 that uses expect.rb.

Oh, wait, do you want to test C++ from Ruby using Test::Unit?
Seems like an interesting way to write tests for C++ code, but
I think the hammer might be bigger than necessary for the nail
you're trying to hit.  ;-)

-- Dossy

--

Panoptic Computer Network             web: http://www.panoptic.com/
  "He realized the fastest way to change is to laugh at your own
    folly -- then you can let go and quickly move on." (p. 70)



Fri, 12 Nov 2004 11:27:22 GMT  
 using test::unit for C++ unit tests

Quote:

>Phil:

>As a C/C++ guy at heart, I have given some thought to this problem (i.e.,
>that of using Ruby to generate unit tests for other languages like, oh, say
>C and C++). In fact, it's one of the core themes of the book I'm working on
>with Rich Kilmer and Dana Moore: Ruby as a software engineering toolsmith
>language.

>One approach (the one toward which I lean) is to generate the C++ classes
>using Ruby as a sort of code specification language. I'm calling the package
>RuGen, and it will have an assortment of code generators one day, but
>upfront I'm working on code generators for Java and C++. The beauty of this
>approach is it becomes unnecessary to parse a C++ file to determine what to
>test, because you already have all of the metadata about the code you need
>to a.) generate the code (or at least the complete skeleton/interface
>thereof) and b.) generate the test code for the generated code, treating the
>interface as a "black box". (I'm writing a separate package, RuLog, to
>provide the logic/inference capabilities necessary to create meaningful and
>comprehensive test cases from the spec).

>RuGen is still in the design/prototype phase, but the idea is fairly clear
>in my head, in terms of how it will work. Rather than create a bunch of
>specific language parsers, have one specification language (in much the same
>way that Ruby/DL is a specification language for dynamic loading of external
>libraries) and a number of generators that are capable of translating the
>spec to code in a more or less plug'n'play, or, if I ever get around to
>slapping a GUI on it, a point'n'click fashion.

>Hopefully I'll have something working soon - I'm probably going to do the
>Java generator first, though. But it should serve as an example to everybody
>else how to build a code generator for the RuGen package (RuGen providing
>the high-level abstractions needed to tie all the plug-in pieces together).

>All thoughts, comments, suggestions welcome.

>:)

Bob,

RuGen sounds really interesting and potentially useful.  So let me see if
I understand correctly: you could write some Ruby classes/code which would
act as an executable spec and then at some point you could translate the
Ruby classes into C++ classes and testcases.  If I'm understanding
correctly, this would be a great way to prototype C++ code using Ruby.
So how will you specify types for the C++ side?

Any idea when we might see the inital release of RuGen?

Phil



Fri, 12 Nov 2004 11:30:12 GMT  
 using test::unit for C++ unit tests

Quote:


>> I'm going to be starting on a C++ development project for a contract I'm
>> doing and I'd like to use the 'test-first' philosophy and develop unit
>> tests first.  Has anyone adapted any of Ruby's unit testing frameworks to
>> do unit testing for C++ classes?

>How about CppUnit?

>  http://cppunit.sourceforge.net/

Someone else just mentioned this.  I'll check it out.

Quote:

>I hear CppUnit is miserable, though.

Anything specific, or is it just miserable in comparison like C++ in
general tends to be 'miserable' to work with compared to Ruby?

Phil



Fri, 12 Nov 2004 12:54:02 GMT  
 using test::unit for C++ unit tests

%
% Bob,
%
% RuGen sounds really interesting and potentially useful.  So let me see if
% I understand correctly: you could write some Ruby classes/code
% which would
% act as an executable spec and then at some point you could translate the
% Ruby classes into C++ classes and testcases.  If I'm understanding
% correctly, this would be a great way to prototype C++ code using Ruby.

You do understand correctly. Except it isn't limited to C++, and the point
is that by merely specifying another generator, you can have the same design
applied (at least in skeletal form) to another language, making it possible
to re-implement the same design in a completely different language by
changing a single variable in the "spec". :)

% So how will you specify types for the C++ side?

Ah, that is the kicker. The problem doesn't stop with just types, but also
extends to things like standard libraries, framework libraries, and GUI
toolkits. The key is to find the right level of abstraction and a "least
common denominator" spec, and then let each generator engine specify those
things unique to its language, platform, etc, in a more or less standard
way. There will be tradeoffs: I don't anticipate being able to cleanly
generate partial template specialization or anything from the Alexandrescu
book on modern C++ design, at least not in release 1.0. ;)

However that isn't to say that the code generated cannot reasonably "depend"
upon some custom C++ library (mirrored in Java or any other language) so
that the commonalities for advanced code generation can be maximized. That
is to say: Some C++ library has to be in place to make various assumptions
about the code to be generated easily replicatable by implementing an
analogous library in some other target language when somebody wants to port
RuGen to that language. Did that make any sense? It places higher demand on
the generator-implementor, but hey, somebody's got to write some "real
code". ;)

If I can just get a basic engine that shows how you can gen a non-trivial
Java and a C++ solution from the same spec, with reasonably complete
functionality, I'll be happy for starters.

The problem has been tackled successfully before: CA's COOL:Gen lets you
take the same data model and generate a complete client-server solution with
the click of a button in 4 or 5 different languages. The code it gens is
really ugly, not at all OO, but it works.

It also costs hundreds of thousands of dollars.

Rational Rose also does something like this with UML, and it goes for $5K a
pop in the Enterprise Edition.

RuGen is a little different because it isn't tied to client-server
relational database design, as COOL:Gen (now Advantage Gen, technically) is,
and it isn't tied to some particular GUI tool, as Rose is. Imagine Ruby as
an alternative to UML for design, where the "model" is really a collection
of Ruby classes - not prototypes, but real design specs - that really do
create both the code and the tests that prove the code works as designed.

That's the Vision Thing, anyway. In the short term, I'll be happy just
proving that the concept can be implemented - in Ruby.
%
% Any idea when we might see the inital release of RuGen?

In about a month or so. The final, official first release however will
probably not be ready until the book is released, only because it will take
me that long to get it done. ;)

-- Bob.

%
% Phil
%



Fri, 12 Nov 2004 13:21:39 GMT  
 using test::unit for C++ unit tests

Quote:

> I'm going to be starting on a C++ development project for a contract I'm
> doing and I'd like to use the 'test-first' philosophy and develop unit
> tests first.  Has anyone adapted any of Ruby's unit testing frameworks to
> do unit testing for C++ classes?

Yes.  Our team does roughly 90% of our C++ testing using Ruby.

Quote:
> I'm thinking that there could be a couple of approaches:
> 1) make the C++ libraries Ruby extensions and instaniate the C++ objects
> and pass messages to them on the Ruby side.

This is the solution we went with.  There are two options:

  a) Write the extensions as standard Ruby extensions, paying careful
  attention to resource management (as our C++ objects are reference
  counted, and Ruby uses a mark-and-sweep GC) as well as exception
  safety (Ruby exceptions must be translated to C++ exceptions and vice
  versa).  This isn't as difficult as it sounds, once you've done it
  once or twice.

  b) Write the extensions using Swig.  For complex templated classes,
  this doesn't seem to work well, but for simple classes, it works
  marvelously.  Exception safety issues still need to be considered, but
  if you can sacrifice a bit of speed, they are certainly less of an
  issue than with writing the extensions by hand.

Quote:
> 2) make some kind of wrapper class (in C++) that gets used to instantiate
> and call methods and then returns some error code.  Compile this wrapper
> into an executable binary that gets called from Ruby with error codes
> being passed back:
> errors = 'exercise <some params>'

I've done this, too.  It's more convenient sometimes, but results in
less maintainable tests.  I find a mix of #1 and #2 to be a reasonable
compromise.

Quote:
> 3) a variation on #2 that uses expect.rb.

I'm not sure what you mean by this.  Can you elaborate?

(Some of our tests for socket-based applications are done using a
variation of expect.rb that is capable of listening for multiple
"chains" of messages simultaneously.  Is this what you are describing?)

Quote:
> While #1 seems perhaps to be the cleanest choice, we don't have much time
> to set all of this up, so creating the extensions might take longer than
> what we'd like.

I don't think any of the above options necessarily has a clear advantage
in terms of time.  What I've found is that getting the first version of
an extension right may take some time, but that time is soon gained back
by a) the development time of the tests is drastically reduced, since
you don't have to recompile the tests whenever they change, and b)
adding a new test is very easy.

Quote:
> Can anyone who has done this offer some pointers?

> Phil

Hope this helps,

Paul



Fri, 12 Nov 2004 17:15:35 GMT  
 using test::unit for C++ unit tests

Quote:


> >I hear CppUnit is miserable, though.

> Anything specific, or is it just miserable in comparison like C++ in
> general tends to be 'miserable' to work with compared to Ruby?

If you stay on the 1.6.x versions, it isn't too bad. From what I've
heard on the forum for it (and from a friend who uses it alot), the
1.8.x version(s) introduce wierd bugs that aren't caught by the tests
for CppUnit itself.

But then having to compile the unittests before running them is kinda
'miserable' just the same :-)

--
(\[ Kent Dahl ]/)_    _~_    __[ http://www.stud.ntnu.no/~kentda/ ]___/~
 ))\_student_/((  \__d L b__/  NTNU - graduate engineering - 4. year  )
( \__\_?|?_/__/ ) _)Industrial economics and technological management(
 \____/_?_\____/ (____engineering.discipline_=_Computer::Technology___)



Fri, 12 Nov 2004 21:24:58 GMT  
 using test::unit for C++ unit tests

Quote:
> >How about CppUnit?

> >  http://cppunit.sourceforge.net/

> Someone else just mentioned this.  I'll check it out.

> >I hear CppUnit is miserable, though.

> Anything specific, or is it just miserable in comparison like C++ in
> general tends to be 'miserable' to work with compared to Ruby?

I can't comment because I haven't used CppUnit, but I recall it's
supposed to be an attempt at a port of JUnit into C++ which doesn't
work very well.  Or something like that.

-- Dossy

--

Panoptic Computer Network             web: http://www.panoptic.com/
  "He realized the fastest way to change is to laugh at your own
    folly -- then you can let go and quickly move on." (p. 70)



Sat, 13 Nov 2004 11:19:30 GMT  
 using test::unit for C++ unit tests
Quote:
>-----Original Message-----

>Sent: Tuesday, 28 May 2002 1:18 PM

>Subject: Re: using test::unit for C++ unit tests

>> >How about CppUnit?

>> >  http://cppunit.sourceforge.net/

>> Someone else just mentioned this.  I'll check it out.

>> >I hear CppUnit is miserable, though.

>> Anything specific, or is it just miserable in comparison like C++ in
>> general tends to be 'miserable' to work with compared to Ruby?

>I can't comment because I haven't used CppUnit, but I recall it's
>supposed to be an attempt at a port of JUnit into C++ which doesn't
>work very well.  Or something like that.

I've used CppUnit, JUnit and RubyUnit.  CppUnit isn't as nice to use as
JUnit, but IMO that is more the fault of C++ than of CppUnit.  There are
macros
to make up for the absence of reflection:  you need to place the macro calls
in
your .hpp & .cpp files to ensure that the tests get linked automatically.

I would much rather use CppUnit than do without test first - and version
1.6.2 was good enough that I didn't want to spend time writing my own tests.

Cheers,

Simon

Simon A. Crase

Invetech Pty Ltd
Private Bag 44
495 Blackburn Road
Mount Waverley  Vic  3149

Phone:  61 3 9211 7933
Mobile: 0408 579 006
Fax:    61 3 9211 7702 (facsimile)

___________________________________________________________
IMPORTANT - This email and any attachments may be confidential. Any
retransmissions, dissemination or other use of these materials by
persons or entities other than the intended recipient is prohibited. If
received in error, please contact us and delete all copies. Before
opening or using attachments, check them for viruses and defects. Our
liability is limited to resupplying any affected attachments. [Any
representations or opinions expressed in this e.mail are those of the
individual sender, and not necessarily those of Vision Systems Limited].



Sat, 13 Nov 2004 11:40:23 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. Test::Unit::Mock: Mock objects for testing with Test::Unit

2. Test order in Test::Unit

3. Test order in Test::Unit

4. Automating UI tests (was Re: Art of Unit Testing)

5. Unit testing data; What to test

6. Test Tool for Unit Tests?

7. Using Test::Unit to assert messages appeared on $stdout/$stde rr

8. Using Test::Unit to assert messages appeared on $stdout/$stderr

9. Using FxRuby and Test::Unit.

10. Unit testing - Using Java Mock Objects in Python?

11. COM and unit tests

12. Report on Unit Testing MVP Presenters.

 

 
Powered by phpBB® Forum Software