Automated Testing Tools for Tcl/Tk 
Author Message
 Automated Testing Tools for Tcl/Tk



Quote:

>> Hey, if you're running a plain ol' normal regular app, it shouldn't matter
>> that it happens to be written in Tk/Tcl -- the standard
>> capture/playback/scripting test tools ought to work fine.  (XRunner, QA
>> Partner, etc.)

>> ATS put "Tk/Tcl" on their page, sure, but I didn't see anything that led me
>> to believe that they were doing anything special over standard
>> capture/playback/scripting.
>> I looked them over after posing the tk/tcl test coverage question on here a
>> month
>> or so ago.  They don't advertise coverage analysis and I'm a little stumped
>> as to what else they might offer that makes them particularly more tk/tcl
>> friendly than any of their competitors.

>> --JMike

>As far as I know, they _are_ doing specific things to allow them
>to capture/palyback Tcl/Tk.  They have a tool called the atstcltalker
>(ats-tcl-talker) which needs to be used to understand tk widgets.
>Currently, it is the only tool out there that I know of that will
>capture/palyback tk GUI applications.  I am in constant contact with
>them reporting problems.  I can say that they have been very good about
>trying to resolve problems.

>I don't think any of the tools you mentioned work with tk.  From what
>I've read, they work mainly with X.

>X-Tester does not do coverage analysis.  It is a capture/playback tool
>only.  But, at least from the research we have done, I don't think they
>have any competitors (at least in the tk automated testing arena).

>I will look more closely at the tools you have mentioned to see if
>they do support tk GUI's.

                        .
                        .
                        .
Two more possibilities:
1.  TkReplay <URL:
    ftp://ftp.cs.unm.edu/pub/crowley/replay/TkReplay1.1.tar.gz>
2.  Tk is completely scriptable.  Tk applications
    can be *designed* to be entirely verifiable.
    This is an exceedingly powerful idea, and one
    that I think is still insufficiently appreci-
    ated.  At the least, lean on the development
    side to design a clean, low-connectivity inter-
    face between the functional parts of the
    application(s) and the GUI.
--

Cameron Laird           http://www.*-*-*.com/ ~claird/home.html



Thu, 06 Apr 2000 03:00:00 GMT  
 Automated Testing Tools for Tcl/Tk


                        .
                        .
                        .

Quote:
>2.  Tk is completely scriptable.  Tk applications
>    can be *designed* to be entirely verifiable.
>    This is an exceedingly powerful idea, and one
>    that I think is still insufficiently appreci-
>    ated.  At the least, lean on the development
>    side to design a clean, low-connectivity inter-
>    face between the functional parts of the
>    application(s) and the GUI.

                        .
                        .
                        .
Correspondents have asked me to move from the declamatory
to the descriptive.  This is indeed a subject about which
I'm passionate, and I'll detail a bit more of what I have
in mind:

Students of this topic should consider three themes:
A.  Study the `make test` supplied with the Tk
    installation.  It provides an existence proof
    for the power that can be achieved by scrip-
    ting Tk tests in Tk.
B.  "Tk is completely scriptable", I wrote above.
    All pointer movements, mouse clicks, ... ger-
    mane to Tk operation are inherently available
    programmatically.  In fact, with Tk's simple
    send mechanism <URL:
    http://www.elf.org/tcltk-man-html/TkCmd/send.htm>
    it's (essentially) always possible to script
    a productized Tk application externally and
    completely.  I'll say this concretely:  QA
    departments can test shrink-wrapped products,
    remotely, and with comprehensive automation.
    For what more could one ask?
C.  Tk also facilitates elements of my personal
    ideology of software development.  I advocate,
    for example, writing user manuals first, *then*
    test plans, *then* implementation (I'm leaving
    out intermediate steps here).  I recognize that
    the industry generally traces a trajectory which
    reverse mine.

    Here's the technical point:  Tk makes what Mark
    Kramer calls "the egg model" realistic.  "The
    egg model" realizes testability by partitioning
    development effort into an inner ("yolk") func-
    tional component, and an outer ("white")
    controlling GUI.  The defect of this metaphor is
    that it insufficiently emphasizes this key point:
    there should be very simple connectivity between
    yolk and white.  The yolk is always testable, as
    the API of the implementing language (Tcl com-
    mands themselves, in the case of Tcl/Tk) controls
    it; the interface to the white is constructed to
    be simple, and therefore robustly testable; and,
    if the GUI elements are scriptable, the QA chal-
    lenge has been decomposed into three eminently
    manageable segments.

These three themes all operate most effectively when the
Development staff has adopted their lessons.  Tk certainly
eases the work of QA in organizations which live the
adversarial chart in which Development does whatever it
wants, and then QA tries to break it; the *serious* multi-
pliers only kick in, though, when QA and Development work
together from the beginning, in co-operation.  With this
sort of co-operation, QA can take responsibility for edu-
cating Development about testability in analysis and design;
the egg model for implementation architecture; and the
technical strengths of Tk (for example) in implementing
testable GUIs.
--

Cameron Laird           http://starbase.neosoft.com/~claird/home.html



Sat, 08 Apr 2000 03:00:00 GMT  
 Automated Testing Tools for Tcl/Tk

Quote:


[snip]
> B.  "Tk is completely scriptable", I wrote above.
>     All pointer movements, mouse clicks, ... ger-
>     mane to Tk operation are inherently available
>     programmatically.  

This is not entirely true. It is not possible to intereact with all
menus using tcl commands. A while back I wrote a truly elegant (ahem
IMHO) record and play function using some of the features of 8.0 alpha.
It was just a few hundred lines  of code total. Then I deleted it by
mistake :-(. Then I recreated it from scratch. Works great. Except that
on Windows It can't record and successfully play back events over
toplevel menubars. Never tried the Mac but I bet the same problem
exists.

There seems to be some bugs related to native menus. If my cursor
physically (so to speak) is over the menu at playback time it *almost*
works. If the mouse is outside the menu (or where the menu will pop up)
it just flat won't. This is with using event generate to generate mouse
motion and button clicks over menubars and menus. I also tried
synthesizing <MenuSelect> or whatever the virtual events are menus use
(I forget now -- it's been a while).

Bottom line was, I could not get to the point where I could automate the
interaction with native menus.

Such a shame, really.

Quote:
>     In fact, with Tk's simple
>     send mechanism <URL:
>     http://www.elf.org/tcltk-man-html/TkCmd/send.htm>
>     it's (essentially) always possible to script
>     a productized Tk application externally and
>     completely.  I'll say this concretely:  QA
>     departments can test shrink-wrapped products,
>     remotely, and with comprehensive automation.
>     For what more could one ask?

For what more could one ask? That this dream be a reality. When I first
tried my record and play code (at the time the core of it was < 100
lines of code!) I was thrilled beyond expectation. Simple and quite
effective. A huge improvement over TkReplay in code size, speed of
execution, and completeness. Yet the reality is, there are still some
problems.

I know I'm not alone on this one -- someone at the same time was trying
to simulate menu actions and failing.

--

Senior GUI Designer               http://www1.clearlight.com/~oakley/
Healthcare Communications, Inc.   http://www.healthcare.com



Sat, 08 Apr 2000 03:00:00 GMT  
 Automated Testing Tools for Tcl/Tk


Quote:


>                    .
>                    .
>                    .
>>2.  Tk is completely scriptable.  Tk applications
>>    can be *designed* to be entirely verifiable.
>>    This is an exceedingly powerful idea, and one
>>    that I think is still insufficiently appreci-
>>    ated.  At the least, lean on the development
>>    side to design a clean, low-connectivity inter-
>>    face between the functional parts of the
>>    application(s) and the GUI.

        I've tried many times to do some GUI testing with some kind of
record/playback utilities on tk but on many occasion, they either
failed to see itcl objects or native Windows Dialogs.

        We're developing all our GUI apps with tcl/tk and are now
porting to tk8.0, but I still haven't found any good tool that would
work.  I checked DEJAGNU Testing Framework but haven't seen ways to
port it on Win32 correctly.

        One word that is true about tcl/tk is, it's great for testing
APIs.  We use a combination of SWIG (Simple Wrapper Interface
Generator) which allowes us to easily build a tcl layer around our
API's to whatever level is required and I use tcl/tk and lists to test
them.   For that, it's pretty efficient.

        If anybody finds a way to trully automate GUI testing of
tcl/tk 8.0 apps, I would really like to know.   But my problem is:
It has to be portable on all platform (Unix and Win32) !

Raymond Rivest



Sat, 15 Apr 2000 03:00:00 GMT  
 Automated Testing Tools for Tcl/Tk

Quote:

>    We're developing all our GUI apps with tcl/tk and are now
> porting to tk8.0, but I still haven't found any good tool that would
> work.  I checked DEJAGNU Testing Framework but haven't seen ways to
> port it on Win32 correctly.

DejaGNU can be modified to work on Win32 as long as your application has a
non-interactive mode.  DejaGNU uses expect to facilitate testing of
interactive applications.  If your application is not interactive, you can
modify DejaGNU to work with pure Tcl.

If anyone's interested, I can (probably) provide patches for DejaGNU.



Sat, 15 Apr 2000 03:00:00 GMT  
 Automated Testing Tools for Tcl/Tk

Quote:

> >       We're developing all our GUI apps with tcl/tk and are now
> > porting to tk8.0, but I still haven't found any good tool that would
> > work.  I checked DEJAGNU Testing Framework but haven't seen ways to
> > port it on Win32 correctly.

> DejaGNU can be modified to work on Win32 as long as your application has a
> non-interactive mode.  DejaGNU uses expect to facilitate testing of
> interactive applications.  If your application is not interactive, you can
> modify DejaGNU to work with pure Tcl.

> If anyone's interested, I can (probably) provide patches for DejaGNU.

The modifications to the expect script are pretty minor.  I just
supplied replacements to the expect functions.  I also made the
scripts run with either expect or not (this worked on Solaris, Linux,
95 and NT).  I don't see a problem with us distributing the changes.

You can also get Expect for NT.  Despite what the FQ says, I believe
that there was an announcement to the effect that there is a windows
port coming.

From the Expect FAQ (http://expect.nist.gov/FAQ.html):

Quote:
> Gordon Chaffee has ported Expect to NT. You can get his port
> from http://bmrc.berkeley.edu/people/chaffee/tcltk.html

> I would like to unify the UNIX and NT code but I probably won't have
> the time to do this personally. Volunteers are welcome.

> I have no plans to do other ports. Feel free to volunteer.

--



Sun, 16 Apr 2000 03:00:00 GMT  
 
 [ 7 post ] 

 Relevant Pages 

1. Automated testing tools for tcl/tk

2. Using TCL/tk/Expect for Automated GUI Testing

3. automated tests of tcl/tk interface

4. Automated Testing for TCL/TK

5. Automated testing of Tcl/Tk applications

6. Automated testing methods of Tcl/Tk programs?

7. VAST - Automated Testing Tools

8. VW 2.5 automated testing tools

9. Automated Smalltalk Testing Tools

10. -Looking for Automated Test Tool

11. Need Automated (Smalltalk) software testing tool

12. VisualAge - Problems using Automated Testing Tool with VisualAge

 

 
Powered by phpBB® Forum Software