data extraction from screen or text file 
Author Message
 data extraction from screen or text file

     I would like to be able to extract data from a screen of a legacy
system.  Are there API's etc that would allow one to do this from a Clarion
app?  Basically I want to search for a pattern of text on screen, like
"Pathology Results:".  Then collect data that comes directly after this
pattern.  This can be done of course if the legacy app can produce a text
file of the data on screen (or some other query) but ideally I would like to
be able to do this from a screen.

    I have done some searching on the Internet for "Screen Scraping",
"Screen Scrubbing", which seem to be the terms for this type of activity.
However most solutions concentrate on doing this with web pages.  The
reasoning behind this is that our main hospital system produces text based
reports.  It will be years before they have a system of discreet data
elements that can be queried.  Thus for QA and research purposes we would
like to be able to extract specific data from on-screen reports.

Thanks.



Mon, 07 Nov 2005 23:48:55 GMT  
 data extraction from screen or text file
I am sure there are better ways, but for simplicity we have tended to
use terminal emulation software that we call from clarion. This is
controlled by it's own scripting language and extracts data to a text file.

We then log out and return control to clarion which processes the text file.

Generally for reports we tend to run the program from the emulator,
saving the report from there to a text file and then ftp the report file
back to the clarion machine for processing. That depends on the host
system being able to output to a spool/text file and being able to
locate it and ftp. Otherwise you can just pick up from the screen.

One reason for using a terminal emulator, is that they are designed for
the task, have pretty good scripting languages and these may be changed
, re-run and tested without having to make changes to clarion code. This
  is important when there is a possibility of the legacy app changing,
and also because flexibility is required as apps designed to be run by
humans are often not that simple to control authomatically.

In terms of emulation software we used TerraTerm Pro initially, it is
freeware, but have changed to TTwin3 which is a commercial product and
more powerful, plus they gave good support.

Quote:

>      I would like to be able to extract data from a screen of a legacy
> system.  Are there API's etc that would allow one to do this from a Clarion
> app?  Basically I want to search for a pattern of text on screen, like
> "Pathology Results:".  Then collect data that comes directly after this
> pattern.  This can be done of course if the legacy app can produce a text
> file of the data on screen (or some other query) but ideally I would like to
> be able to do this from a screen.

>     I have done some searching on the Internet for "Screen Scraping",
> "Screen Scrubbing", which seem to be the terms for this type of activity.
> However most solutions concentrate on doing this with web pages.  The
> reasoning behind this is that our main hospital system produces text based
> reports.  It will be years before they have a system of discreet data
> elements that can be queried.  Thus for QA and research purposes we would
> like to be able to extract specific data from on-screen reports.

> Thanks.



Tue, 08 Nov 2005 01:01:20 GMT  
 data extraction from screen or text file
someone mentioned the SnagIt utility a couple of days ago as a way of converting
screens to usable text.  Might serve the need
http://www.techsmith.com/products/snagit/features.asp
        Tim
Quote:

>      I would like to be able to extract data from a screen of a legacy
> system.  Are there API's etc that would allow one to do this from a Clarion
> app?  Basically I want to search for a pattern of text on screen, like
> "Pathology Results:".  Then collect data that comes directly after this
> pattern.  This can be done of course if the legacy app can produce a text
> file of the data on screen (or some other query) but ideally I would like to
> be able to do this from a screen.

>     I have done some searching on the Internet for "Screen Scraping",
> "Screen Scrubbing", which seem to be the terms for this type of activity.
> However most solutions concentrate on doing this with web pages.  The
> reasoning behind this is that our main hospital system produces text based
> reports.  It will be years before they have a system of discreet data
> elements that can be queried.  Thus for QA and research purposes we would
> like to be able to extract specific data from on-screen reports.

> Thanks.



Tue, 08 Nov 2005 01:11:30 GMT  
 data extraction from screen or text file
Thanks for the two suggestions.  I have looked at both.  SnagIt has a COM
server so I think I'll see what I can do with it.  Now I just need to get up
to speed with COM.  I have the PlugWare COM classes and OutLookFuse but
can't seem to get my brain wrapped around this stuff.  Anyone have any
similar experiences implementing COM into Clarion apps?

Thanks


Quote:
>      I would like to be able to extract data from a screen of a legacy
> system.  Are there API's etc that would allow one to do this from a
Clarion
> app?  Basically I want to search for a pattern of text on screen, like
> "Pathology Results:".  Then collect data that comes directly after this
> pattern.  This can be done of course if the legacy app can produce a text
> file of the data on screen (or some other query) but ideally I would like
to
> be able to do this from a screen.

>     I have done some searching on the Internet for "Screen Scraping",
> "Screen Scrubbing", which seem to be the terms for this type of activity.
> However most solutions concentrate on doing this with web pages.  The
> reasoning behind this is that our main hospital system produces text based
> reports.  It will be years before they have a system of discreet data
> elements that can be queried.  Thus for QA and research purposes we would
> like to be able to extract specific data from on-screen reports.

> Thanks.

-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----==  Over 80,000 Newsgroups - 16 Different Servers! =-----


Tue, 08 Nov 2005 10:52:38 GMT  
 data extraction from screen or text file
I must have misunderstood your need then, I thought you would need to
actually control the legacy app from the Clarion System in order to get
the information you wished to capture and then deal with it
automatically, that is my understanding of screen scraping.

Have fun.

Quote:

> Thanks for the two suggestions.  I have looked at both.  SnagIt has a COM
> server so I think I'll see what I can do with it.  Now I just need to get up
> to speed with COM.  I have the PlugWare COM classes and OutLookFuse but
> can't seem to get my brain wrapped around this stuff.  Anyone have any
> similar experiences implementing COM into Clarion apps?

> Thanks



>>     I would like to be able to extract data from a screen of a legacy
>>system.  Are there API's etc that would allow one to do this from a

> Clarion

>>app?  Basically I want to search for a pattern of text on screen, like
>>"Pathology Results:".  Then collect data that comes directly after this
>>pattern.  This can be done of course if the legacy app can produce a text
>>file of the data on screen (or some other query) but ideally I would like

> to

>>be able to do this from a screen.

>>    I have done some searching on the Internet for "Screen Scraping",
>>"Screen Scrubbing", which seem to be the terms for this type of activity.
>>However most solutions concentrate on doing this with web pages.  The
>>reasoning behind this is that our main hospital system produces text based
>>reports.  It will be years before they have a system of discreet data
>>elements that can be queried.  Thus for QA and research purposes we would
>>like to be able to extract specific data from on-screen reports.

>>Thanks.

> -----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
> http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
> -----==  Over 80,000 Newsgroups - 16 Different Servers! =-----



Tue, 08 Nov 2005 13:44:19 GMT  
 data extraction from screen or text file
Shaun,

    That is what I want to do, control the capture of screen text from a
Clarion program.  How would one use a terminal emulation program to do this?
The screens I want to "scrub" are from a running Windows app.  Using DDE I
can do some of this with SnagIt but not sure how I would use TeraTerm to do
this.

Thanks

Jeff King


Quote:
> I must have misunderstood your need then, I thought you would need to
> actually control the legacy app from the Clarion System in order to get
> the information you wished to capture and then deal with it
> automatically, that is my understanding of screen scraping.

> Have fun.


> > Thanks for the two suggestions.  I have looked at both.  SnagIt has a
COM
> > server so I think I'll see what I can do with it.  Now I just need to
get up
> > to speed with COM.  I have the PlugWare COM classes and OutLookFuse but
> > can't seem to get my brain wrapped around this stuff.  Anyone have any
> > similar experiences implementing COM into Clarion apps?

> > Thanks



> >>     I would like to be able to extract data from a screen of a legacy
> >>system.  Are there API's etc that would allow one to do this from a

> > Clarion

> >>app?  Basically I want to search for a pattern of text on screen, like
> >>"Pathology Results:".  Then collect data that comes directly after this
> >>pattern.  This can be done of course if the legacy app can produce a
text
> >>file of the data on screen (or some other query) but ideally I would
like

> > to

> >>be able to do this from a screen.

> >>    I have done some searching on the Internet for "Screen Scraping",
> >>"Screen Scrubbing", which seem to be the terms for this type of
activity.
> >>However most solutions concentrate on doing this with web pages.  The
> >>reasoning behind this is that our main hospital system produces text
based
> >>reports.  It will be years before they have a system of discreet data
> >>elements that can be queried.  Thus for QA and research purposes we
would
> >>like to be able to extract specific data from on-screen reports.

> >>Thanks.

> > -----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
> > http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
> > -----==  Over 80,000 Newsgroups - 16 Different Servers! =-----

-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----==  Over 80,000 Newsgroups - 16 Different Servers! =-----


Thu, 10 Nov 2005 21:31:12 GMT  
 data extraction from screen or text file
Jeff,

JAT, I was looking at this several years ago when trying to interface
with a CICS mainfram application.  At that time I was looking for a
terminal emulation program that supported IBM's HLLAPI (High Level
Language API).  Using HLLAPI (an spec for interfacing with IBMs terminal
emulation programs) you can pretty much control the way the application
running on the terminal emulator works and get/put data into the various
fields displayed on the emulation screen.  

I also found several programs that would run "test scripts" against any
windows program.  These programs are useful when testing and performing
various regression analysis on program changes.   I also considered
writing a program to generate the test script (an ASCII file) but then
the client cancelled the project and I stopped working on the concept.

Don't know if this is what you're trying to accomplish or not so may not
be applicable.

Just couple of other ideas.

Al

Quote:

> Shaun,

>     That is what I want to do, control the capture of screen text from a
> Clarion program.  How would one use a terminal emulation program to do this?
> The screens I want to "scrub" are from a running Windows app.  Using DDE I
> can do some of this with SnagIt but not sure how I would use TeraTerm to do
> this.

> Thanks

> Jeff King



> > I must have misunderstood your need then, I thought you would need to
> > actually control the legacy app from the Clarion System in order to get
> > the information you wished to capture and then deal with it
> > automatically, that is my understanding of screen scraping.

> > Have fun.


> > > Thanks for the two suggestions.  I have looked at both.  SnagIt has a
> COM
> > > server so I think I'll see what I can do with it.  Now I just need to
> get up
> > > to speed with COM.  I have the PlugWare COM classes and OutLookFuse but
> > > can't seem to get my brain wrapped around this stuff.  Anyone have any
> > > similar experiences implementing COM into Clarion apps?

> > > Thanks



> > >>     I would like to be able to extract data from a screen of a legacy
> > >>system.  Are there API's etc that would allow one to do this from a

> > > Clarion

> > >>app?  Basically I want to search for a pattern of text on screen, like
> > >>"Pathology Results:".  Then collect data that comes directly after this
> > >>pattern.  This can be done of course if the legacy app can produce a
> text
> > >>file of the data on screen (or some other query) but ideally I would
> like

> > > to

> > >>be able to do this from a screen.

> > >>    I have done some searching on the Internet for "Screen Scraping",
> > >>"Screen Scrubbing", which seem to be the terms for this type of
> activity.
> > >>However most solutions concentrate on doing this with web pages.  The
> > >>reasoning behind this is that our main hospital system produces text
> based
> > >>reports.  It will be years before they have a system of discreet data
> > >>elements that can be queried.  Thus for QA and research purposes we
> would
> > >>like to be able to extract specific data from on-screen reports.

> > >>Thanks.

> > > -----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
> > > http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
> > > -----==  Over 80,000 Newsgroups - 16 Different Servers! =-----

> -----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
> http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
> -----==  Over 80,000 Newsgroups - 16 Different Servers! =-----



Sat, 12 Nov 2005 02:04:04 GMT  
 data extraction from screen or text file
Al,

    In my case I want to extract data from a window in another running
Windows app.  Can a terminal emulator be configured to run another Windows
app?  Then run the scripts to get the data from the screen displayed in the
terminal emulator?

Thanks


Quote:
> Jeff,

> JAT, I was looking at this several years ago when trying to interface
> with a CICS mainfram application.  At that time I was looking for a
> terminal emulation program that supported IBM's HLLAPI (High Level
> Language API).  Using HLLAPI (an spec for interfacing with IBMs terminal
> emulation programs) you can pretty much control the way the application
> running on the terminal emulator works and get/put data into the various
> fields displayed on the emulation screen.

> I also found several programs that would run "test scripts" against any
> windows program.  These programs are useful when testing and performing
> various regression analysis on program changes.   I also considered
> writing a program to generate the test script (an ASCII file) but then
> the client cancelled the project and I stopped working on the concept.

> Don't know if this is what you're trying to accomplish or not so may not
> be applicable.

> Just couple of other ideas.

> Al


> > Shaun,

> >     That is what I want to do, control the capture of screen text from a
> > Clarion program.  How would one use a terminal emulation program to do
this?
> > The screens I want to "scrub" are from a running Windows app.  Using DDE
I
> > can do some of this with SnagIt but not sure how I would use TeraTerm to
do
> > this.

> > Thanks

> > Jeff King



> > > I must have misunderstood your need then, I thought you would need to
> > > actually control the legacy app from the Clarion System in order to
get
> > > the information you wished to capture and then deal with it
> > > automatically, that is my understanding of screen scraping.

> > > Have fun.


> > > > Thanks for the two suggestions.  I have looked at both.  SnagIt has
a
> > COM
> > > > server so I think I'll see what I can do with it.  Now I just need
to
> > get up
> > > > to speed with COM.  I have the PlugWare COM classes and OutLookFuse
but
> > > > can't seem to get my brain wrapped around this stuff.  Anyone have
any
> > > > similar experiences implementing COM into Clarion apps?

> > > > Thanks



> > > >>     I would like to be able to extract data from a screen of a
legacy
> > > >>system.  Are there API's etc that would allow one to do this from a

> > > > Clarion

> > > >>app?  Basically I want to search for a pattern of text on screen,
like
> > > >>"Pathology Results:".  Then collect data that comes directly after
this
> > > >>pattern.  This can be done of course if the legacy app can produce a
> > text
> > > >>file of the data on screen (or some other query) but ideally I would
> > like

> > > > to

> > > >>be able to do this from a screen.

> > > >>    I have done some searching on the Internet for "Screen
Scraping",
> > > >>"Screen Scrubbing", which seem to be the terms for this type of
> > activity.
> > > >>However most solutions concentrate on doing this with web pages.
The
> > > >>reasoning behind this is that our main hospital system produces text
> > based
> > > >>reports.  It will be years before they have a system of discreet
data
> > > >>elements that can be queried.  Thus for QA and research purposes we
> > would
> > > >>like to be able to extract specific data from on-screen reports.

> > > >>Thanks.

> > > > -----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
> > > > http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
> > > > -----==  Over 80,000 Newsgroups - 16 Different Servers! =-----

> > -----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
> > http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
> > -----==  Over 80,000 Newsgroups - 16 Different Servers! =-----

-----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
-----==  Over 80,000 Newsgroups - 16 Different Servers! =-----


Sat, 12 Nov 2005 12:54:58 GMT  
 data extraction from screen or text file
if you log all data (from terminal) into file, you can easily examine
content of the file, don't?

Quote:

> Al,

>     In my case I want to extract data from a window in another running
> Windows app.  Can a terminal emulator be configured to run another
> Windows app?  Then run the scripts to get the data from the screen
> displayed in the terminal emulator?

- 16 Different Servers! =-----


Sat, 12 Nov 2005 13:09:50 GMT  
 data extraction from screen or text file
Jeff,

I doubt the terminal emulator would be the way to go since it's dealing
with just emulationg a "dumb" mainframe terminal.  On the other hand
there are scheduling and testing automation programs that DO run other
windows programs so clearly it can be done.  Don't have much more than
that to offer.  

Have you contacted the author of the other program to see what they
might suggest?  Or possibly contact one of the Programmer's
Paradise-type companies to see if they might have a suggestion.

Al

Quote:

> Al,

>     In my case I want to extract data from a window in another running
> Windows app.  Can a terminal emulator be configured to run another Windows
> app?  Then run the scripts to get the data from the screen displayed in the
> terminal emulator?

> Thanks



> > Jeff,

> > JAT, I was looking at this several years ago when trying to interface
> > with a CICS mainfram application.  At that time I was looking for a
> > terminal emulation program that supported IBM's HLLAPI (High Level
> > Language API).  Using HLLAPI (an spec for interfacing with IBMs terminal
> > emulation programs) you can pretty much control the way the application
> > running on the terminal emulator works and get/put data into the various
> > fields displayed on the emulation screen.

> > I also found several programs that would run "test scripts" against any
> > windows program.  These programs are useful when testing and performing
> > various regression analysis on program changes.   I also considered
> > writing a program to generate the test script (an ASCII file) but then
> > the client cancelled the project and I stopped working on the concept.

> > Don't know if this is what you're trying to accomplish or not so may not
> > be applicable.

> > Just couple of other ideas.

> > Al


> > > Shaun,

> > >     That is what I want to do, control the capture of screen text from a
> > > Clarion program.  How would one use a terminal emulation program to do
> this?
> > > The screens I want to "scrub" are from a running Windows app.  Using DDE
> I
> > > can do some of this with SnagIt but not sure how I would use TeraTerm to
> do
> > > this.

> > > Thanks

> > > Jeff King



> > > > I must have misunderstood your need then, I thought you would need to
> > > > actually control the legacy app from the Clarion System in order to
> get
> > > > the information you wished to capture and then deal with it
> > > > automatically, that is my understanding of screen scraping.

> > > > Have fun.


> > > > > Thanks for the two suggestions.  I have looked at both.  SnagIt has
> a
> > > COM
> > > > > server so I think I'll see what I can do with it.  Now I just need
> to
> > > get up
> > > > > to speed with COM.  I have the PlugWare COM classes and OutLookFuse
> but
> > > > > can't seem to get my brain wrapped around this stuff.  Anyone have
> any
> > > > > similar experiences implementing COM into Clarion apps?

> > > > > Thanks



> > > > >>     I would like to be able to extract data from a screen of a
> legacy
> > > > >>system.  Are there API's etc that would allow one to do this from a

> > > > > Clarion

> > > > >>app?  Basically I want to search for a pattern of text on screen,
> like
> > > > >>"Pathology Results:".  Then collect data that comes directly after
> this
> > > > >>pattern.  This can be done of course if the legacy app can produce a
> > > text
> > > > >>file of the data on screen (or some other query) but ideally I would
> > > like

> > > > > to

> > > > >>be able to do this from a screen.

> > > > >>    I have done some searching on the Internet for "Screen
> Scraping",
> > > > >>"Screen Scrubbing", which seem to be the terms for this type of
> > > activity.
> > > > >>However most solutions concentrate on doing this with web pages.
> The
> > > > >>reasoning behind this is that our main hospital system produces text
> > > based
> > > > >>reports.  It will be years before they have a system of discreet
> data
> > > > >>elements that can be queried.  Thus for QA and research purposes we
> > > would
> > > > >>like to be able to extract specific data from on-screen reports.

> > > > >>Thanks.

> > > > > -----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
> > > > > http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
> > > > > -----==  Over 80,000 Newsgroups - 16 Different Servers! =-----

> > > -----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
> > > http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
> > > -----==  Over 80,000 Newsgroups - 16 Different Servers! =-----

> -----= Posted via Newsfeeds.Com, Uncensored Usenet News =-----
> http://www.newsfeeds.com - The #1 Newsgroup Service in the World!
> -----==  Over 80,000 Newsgroups - 16 Different Servers! =-----



Sat, 12 Nov 2005 21:57:01 GMT  
 
 [ 10 post ] 

 Relevant Pages 

1. File Data Extraction Approach

2. tps data extraction or data recovery

3. Stopping off screen data in a text widget?

4. Screen data entry box that displays data from text file AND write back to file.

5. How to convert text file to data file?

6. Text block extraction script required

7. Data Extraction

8. Sending a text file to screen on exit?

9. Extraction of data from Clarion 2.1 for DOS to Access 97

10. Text Extraction Tool, Part 2

11. Showing real big text files on the screen

12. Help please/Data extraction from Lotus for clipper

 

 
Powered by phpBB® Forum Software