are there coverage analysis tools? 
Author Message
 are there coverage analysis tools?

I've found unit testing with coverage analysis to check the quality of the
test suite to be a highly useful paradigm and would like to use it in
Eiffel.  I don't just mean e.g. a profiler (which only gives statement
coverage); I mean branch coverage and conditional coverage.  E.g. if you
have an if statement
if (a or b)
the coverage analyzer can tell you that you've executed it with a being
true, with b being true, and with neither being true.

A couple years ago, the lack of such tools was a factor in my reluctantly
getting turned off by Eiffel and feeling like its user base is still too
small to support such tools yet... I think this is one reason people stick
with C/C++ (not because the languages themselves are necessarily great, but
because such a large number of useful tools exist).

I recently am re-investigating Eiffel, and I am still unable to find any
coverage analysis tools for Eiffel (which surprises me).  Do any exist?
It's frustrating to have such a pleasing language with such a lack of tools.
(I'd also like a static analysis tool analogous to lint, which can warn
about things like defining but not using a variable, assigning a variable
but not reading from it, etc., but none seems to exist.)

russ
email = russ at u d dot com



Sat, 18 Dec 2004 00:57:25 GMT  
 are there coverage analysis tools?

Quote:

> I recently am re-investigating Eiffel, and I am still unable to find any
> coverage analysis tools for Eiffel (which surprises me).  Do any exist?

Try:

        http://metrics.eiffel.com/

Let us know if they do what you want!

Regards,

Berend. (-:



Sat, 18 Dec 2004 07:13:27 GMT  
 are there coverage analysis tools?



Quote:

> > I recently am re-investigating Eiffel, and I am still unable to find any
> > coverage analysis tools for Eiffel (which surprises me).  Do any exist?

> Try:

> http://metrics.eiffel.com/

> Let us know if they do what you want!

Sadly, no.  I'd found that page while searching earlier, but those are all
static metrics, i.e. properties of the source code itself (how many features
in these classes, how many classes, etc.) which are indeed interesting but
not what I'm talking about. :)

I'm talking about properties of an execution as opposed to properties of the
source code.  E.g. on this run, how many features got used?  Which lines of
code were hit?  Which if-statements did not get taken with both the true and
the false branches?  Etc.  This sort of info is normally gathered by
automatically instrumenting the source code before it's compiled; the
instrumentation code then gathers the statistics as your program executes.
There are plenty of such tools for other languages, and it's weird that I'm
not finding any for Eiffel. :(  I've had a preliminary look at the
smalleiffel compiler and am wondering how hard it would be to modify it to
generate instrumentation in the C code it produces...

thanks,
Russ



Wed, 22 Dec 2004 02:00:04 GMT  
 are there coverage analysis tools?
On Fri, 5 Jul 2002 13:00:04 -0500, "Russ Williams"

Quote:




>> > I recently am re-investigating Eiffel, and I am still unable to find any
>> > coverage analysis tools for Eiffel (which surprises me).  Do any exist?

>> Try:

>> http://metrics.eiffel.com/

>> Let us know if they do what you want!

>Sadly, no.  I'd found that page while searching earlier, but those are all
>static metrics, i.e. properties of the source code itself (how many features
>in these classes, how many classes, etc.) which are indeed interesting but
>not what I'm talking about. :)

>I'm talking about properties of an execution as opposed to properties of the
>source code.  E.g. on this run, how many features got used?  Which lines of
>code were hit?  Which if-statements did not get taken with both the true and
>the false branches?  Etc.  This sort of info is normally gathered by
>automatically instrumenting the source code before it's compiled; the
>instrumentation code then gathers the statistics as your program executes.
>There are plenty of such tools for other languages, and it's weird that I'm
>not finding any for Eiffel. :(  I've had a preliminary look at the
>smalleiffel compiler and am wondering how hard it would be to modify it to
>generate instrumentation in the C code it produces...

When you GCC that this will not be more then a few hours (if you now
perl). Use the gcc -coverage option to compile the c files do a line
coverage on that and then write a little perl script that merges the c
line back to the eiffel line.

But GCC only offers line coverage, not statement or conditional
coverage.



Wed, 22 Dec 2004 07:53:36 GMT  
 are there coverage analysis tools?

Quote:




>> > I recently am re-investigating Eiffel, and I am still unable to find any
>> > coverage analysis tools for Eiffel (which surprises me).  Do any exist?

>> Try:

>> http://metrics.eiffel.com/

>> Let us know if they do what you want!

>Sadly, no.  I'd found that page while searching earlier, but those are all
>static metrics, i.e. properties of the source code itself (how many features
>in these classes, how many classes, etc.) which are indeed interesting but
>not what I'm talking about. :)

>I'm talking about properties of an execution as opposed to properties of the
>source code.  E.g. on this run, how many features got used?  Which lines of
>code were hit?  Which if-statements did not get taken with both the true and
>the false branches?  Etc.  This sort of info is normally gathered by

Doesn't the ISE Eiffel toolset have a profiling tool that is capable of at
least some of these run-time statistics?

Quote:
>automatically instrumenting the source code before it's compiled; the
>instrumentation code then gathers the statistics as your program executes.
>There are plenty of such tools for other languages, and it's weird that I'm
>not finding any for Eiffel. :(  I've had a preliminary look at the
>smalleiffel compiler and am wondering how hard it would be to modify it to
>generate instrumentation in the C code it produces...

>thanks,
>Russ

--
Jim Cochrane

[When responding by email, include the term non-spam in the subject line to
get through my spam filter.]



Wed, 22 Dec 2004 04:38:26 GMT  
 are there coverage analysis tools?

Quote:

> When you GCC that this will not be more then a few hours (if you now
> perl). Use the gcc -coverage option to compile the c files do a line
> coverage on that and then write a little perl script that merges the c
> line back to the eiffel line.

You probably can use the #LINE directory (or something like that, don't
know exactly how it is called).

But I've not been successfull in compiling SE programs with -pg option.
Just got a random core dump.

Regards,

Berend. (-:



Fri, 24 Dec 2004 04:10:22 GMT  
 are there coverage analysis tools?

Quote:

> But I've not been successfull in compiling SE programs with -pg option.
> Just got a random core dump.

I use SE with gcc and sometimes use -pg. I add -g and -no_strip too. This
works fine.

--
Philippe Ribet

                         The README file said
              "Requires Windows 95, NT 4.0, or better."
                    So... I installed it on Linux!



Fri, 24 Dec 2004 16:11:33 GMT  
 are there coverage analysis tools?

Quote:

> I use SE with gcc and sometimes use -pg. I add -g and -no_strip too. This
> works fine.

Thanks, I'll try this next time.

Regards,

Berend. (-:



Sat, 25 Dec 2004 05:35:43 GMT  
 are there coverage analysis tools?

Quote:

> Doesn't the ISE Eiffel toolset have a profiling tool that is capable of at
> least some of these run-time statistics?

The ISE toolset provides an interesting profiling tool.
It records, for a particular program execution
(a) execution time of each called feature (own time, callees time)
(b) number of times a feature has been called

This is a very good performance analysis tool.

Unfortunately, it does not record which branches of "if/then/else" or
"inspect/when/else" have been visited.

However, by reorganizing sensible if/then/else and inspect/when/else
instructions the code coverage of an application could be measured by
such a tool.

For example
 ...
 if test then
    <code section 1>
 else
    <code section 2>
 end
 ...
could be "code coveraged" by converting it to

    if test then
       do_code_section_1
    else
       do_code_section_2
    end

 ...
 do_code_section_1 is do <code section 1> end
 do_code_section_2 is do <code section 2> end
 ...

There also are other options like making the Eiffel compiler generate
#line directives in the C code.  This way you can benefit from other
tools.

Hope this helps,
Best regards,

Paul G. Crismer



Sat, 25 Dec 2004 21:38:32 GMT  
 are there coverage analysis tools?
Yeah, I'm also thinking my best first option is to try doing coverage
analysis on the generated C code.

thanks for the answers,
russ



Sun, 26 Dec 2004 05:18:28 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. Coverage analysis tool from Smalltalk Report Oct/95

2. Looking for a test coverage analysis tool

3. coverage analysis tools

4. free Fortran Coverage Analysis Tool

5. FORTRAN Test Coverage Analysis Tools wanted

6. Test coverage analysis tool

7. Is there a test coverage analysis tool for tcl-code

8. Any tool for code coverage analysis on Python program

9. Test coverage analysis for pure functional languages?

10. Request info about Coverage analysis

11. code coverage analysis

12. Request info about Coverage analysis

 

 
Powered by phpBB® Forum Software