OS/2 classic REXX, SysTempFileName 
Author Message
 OS/2 classic REXX, SysTempFileName

 Hi,
Apologies for crossposting

Can anyone take a guess as to what is causing me to get a sys1083.

which results in sys1083.

SysTempFileName does not seem to produce the correct results.

I could change it to first set a temporary file using eg.
tf = path||SysTempFileName('?????.FWD')
then run the rename command.

Would it make any difference?

==================

The directory in which the temporary file is created could have over
7k files similarly created.  There is a "known" problem in os/2
where it cannot cope with too many (how many?) files in one
directory.  Could that have anything to do with it?

I could increase the name of the file from 5 questionmarks to more
by generating random letters, eg.  SysTempFileName('?????****.FWD')
where * is a random letter.  Not sure what difference it might make.

Any thoughts appreciated.
Regards
Charles

--

remove "_removeme" to reply.
-------------------------------------------------------------------
Charles Christacopoulos, Secretary's Office, University of Dundee,
Dundee DD1 4HN, (Scotland) United Kingdom.
Tel: +44+(0)1382-344891. Fax: +44+(0)1382-201604.
http://www.*-*-*.com/
Scottish Search Maestro http://www.*-*-*.com/
Win 2000? Still happy using OS2 Warp and so do the HTTPs above :-)



Sat, 04 Dec 2004 00:05:11 GMT  
 OS/2 classic REXX, SysTempFileName

Quote:

> Hi,
>Apologies for crossposting

>Can anyone take a guess as to what is causing me to get a sys1083.
>I have the following line in a rexx program:

>which results in sys1083.

You probably have the wrong syntax for RENAME. You can't put a drive letter
on the target name.

Quote:
>SysTempFileName does not seem to produce the correct results.

Why do you say that?

What happens when you run? :-

say SysTempFileName('?????.FWD')

Quote:
>Any thoughts appreciated.
>Regards
>Charles

>--

>remove "_removeme" to reply.
>-------------------------------------------------------------------
>Charles Christacopoulos, Secretary's Office, University of Dundee,

--
John


Sat, 04 Dec 2004 02:00:30 GMT  
 OS/2 classic REXX, SysTempFileName
hi

 > Can anyone take a guess as to what is causing me to get a sys1083.

Quote:
> I have the following line in a rexx program:

> which results in sys1083.

> SysTempFileName does not seem to produce the correct results.

help 1083 will tell you, that he error is related to the Rename-command.

Quote:

> I could change it to first set a temporary file using eg.
> tf = path||SysTempFileName('?????.FWD')
> then run the rename command.

> Would it make any difference?

Rexx.inf has the following:
"SysTempFileName uses a random number algorithm to determine a number to
replace the filter characters with.  It then tests if the resulting file
exists.  If it does not, it increments the number, replaces the filter
characters, and tests for the existence of the new file, and so on.  This
continues until a free file is found or all possibilities have been
exhausted."
I think this change would secure a valid filename when rename are run.

Quote:

> The directory in which the temporary file is created could have over
> 7k files similarly created.

With that many, and if the random-nature of the names are not mandatory, I
would consider running the Rename in a procedure with a simple count, and let
the count be part of the new filename.

--
good luck

peter



Sat, 04 Dec 2004 05:47:21 GMT  
 OS/2 classic REXX, SysTempFileName

Quote:


> >which results in sys1083.

> You probably have the wrong syntax for RENAME. You can't put a drive letter
> on the target name.

No.  It produced 7000 (or a large number of files) and then it fails.

Quote:

> >SysTempFileName does not seem to produce the correct results.

> Why do you say that?

> What happens when you run? :-

> say SysTempFileName('?????.FWD')

It is a random error.  When it  starts to happen it would do 10 then fail, do 5
then fail.

Thanks
Charles

--

remove "_removeme" to reply.
-------------------------------------------------------------------
Charles Christacopoulos, Secretary's Office, University of Dundee,
Dundee DD1 4HN, (Scotland) United Kingdom.
Tel: +44+(0)1382-344891. Fax: +44+(0)1382-201604.
http://somis.ais.dundee.ac.uk/
Scottish Search Maestro http://somis2.ais.dundee.ac.uk/
Win 2000? Still happy using OS2 Warp and so do the HTTPs above :-)



Sun, 05 Dec 2004 03:20:56 GMT  
 OS/2 classic REXX, SysTempFileName

Quote:

> With that many, and if the random-nature of the names are not mandatory, I
> would consider running the Rename in a procedure with a simple count, and let
> the count be part of the new filename.

> --
> good luck

> peter

Good idea.  Thinking about it, I don't need to even create fully random names,
all I have to watch out for is that each possible process creates its own version
of "random" names and then a simple counter does the job for me.

Problem solved although bug not discovered in full.
Charles

--

remove "_removeme" to reply.
-------------------------------------------------------------------
Charles Christacopoulos, Secretary's Office, University of Dundee,
Dundee DD1 4HN, (Scotland) United Kingdom.
Tel: +44+(0)1382-344891. Fax: +44+(0)1382-201604.
http://somis.ais.dundee.ac.uk/
Scottish Search Maestro http://somis2.ais.dundee.ac.uk/
Win 2000? Still happy using OS2 Warp and so do the HTTPs above :-)



Sun, 05 Dec 2004 03:24:43 GMT  
 OS/2 classic REXX, SysTempFileName

Quote:



>>>which results in sys1083.

>>You probably have the wrong syntax for RENAME. You can't put a drive letter
>>on the target name.

>No.  It produced 7000 (or a large number of files) and then it fails.

>>>SysTempFileName does not seem to produce the correct results.

>>Why do you say that?

>>What happens when you run? :-

>>say SysTempFileName('?????.FWD')

>It is a random error.  When it  starts to happen it would do 10 then fail, do 5
>then fail.

Can you post your code?

I've just created a simple program to create 50,000 files using
SysTempFileName
and it seems OK so far after 15,000...

It seems pointless do a RENAME when you can create the correct filename
in the first
place. It should be much quickerif you are processing a lot of files.

Quote:
>Thanks
>Charles

>--

>remove "_removeme" to reply.
>-------------------------------------------------------------------
>Charles Christacopoulos, Secretary's Office, University of Dundee,

--
John


Sun, 05 Dec 2004 04:32:55 GMT  
 OS/2 classic REXX, SysTempFileName


%

% > >which results in sys1083.
% > >
% > >
% >
% > You probably have the wrong syntax for RENAME. You can't put a drive letter
% > on the target name.
%
% No.  It produced 7000 (or a large number of files) and then it fails.

Have you tried with SysMoveObject? This should rename the file if you pass
it two file names and would remove the command processor from the equation.

I'm not sure what sys1083 is, but if the problem is that the file already
exists, is there some other application running at the same time which
might generate a file with the same name? In that case, you have a race
condition which you might need a semaphore to avoid.

In any case, if you're generating on the order of thousands of files,
SysTempFileName is probably not the most efficient way of ensuring
unique file names. You might want to just keep a sequence number and
store it in an ini file between runs.

--

Patrick TJ McPhee
East York  Canada



Sun, 05 Dec 2004 11:35:32 GMT  
 OS/2 classic REXX, SysTempFileName

Quote:

> It seems pointless do a RENAME when you can create the correct filename
> in the first place.

I don't think you can really say that when you don't know the context.

I use a similar approach when I have a process scanning a directory for
'work'. It only picks up files with a certain form of name; those not
conforming to that are left alone, because they are still being created
or modified.

--
Bob Eager
rde at tavi.co.uk
PC Server 325; PS/2s 8595*3, 9595*3 (2*P60 + P90), 8535, 8570, 9556*2,
8580*6,
8557*2, 8550, 9577, 8530, P70, PC/AT..



Sun, 05 Dec 2004 15:03:00 GMT  
 OS/2 classic REXX, SysTempFileName

Quote:


>>It seems pointless do a RENAME when you can create the correct filename
>>in the first place.

>I don't think you can really say that when you don't know the context.

>I use a similar approach when I have a process scanning a directory for
>'work'. It only picks up files with a certain form of name; those not
>conforming to that are left alone, because they are still being created
>or modified.

 From the original msg, there was the line:-

To me, it looks as though 'TmpFile' is a newly created file which is
subsequently renamed thereby using and extra system call.

There may be some other context, but that is what I suspect is happening.

--
John



Sun, 05 Dec 2004 16:29:18 GMT  
 OS/2 classic REXX, SysTempFileName
Hi,

I am not sure at all but since sys1083 means that "the" file (whatever
file) either already exists, could not be found or is already in use, it
might be a problem of the order of evaluating the REXX statement (who
knows ?).

What happens if you do this:

?

Lars

Charles Christacopoulos schrieb:

Quote:

>  Hi,
> Apologies for crossposting

> Can anyone take a guess as to what is causing me to get a sys1083.
> I have the following line in a rexx program:

> which results in sys1083.

> SysTempFileName does not seem to produce the correct results.

> I could change it to first set a temporary file using eg.
> tf = path||SysTempFileName('?????.FWD')
> then run the rename command.

> Would it make any difference?

> ==================

> The directory in which the temporary file is created could have over
> 7k files similarly created.  There is a "known" problem in os/2
> where it cannot cope with too many (how many?) files in one
> directory.  Could that have anything to do with it?

> I could increase the name of the file from 5 questionmarks to more
> by generating random letters, eg.  SysTempFileName('?????****.FWD')
> where * is a random letter.  Not sure what difference it might make.

> Any thoughts appreciated.
> Regards
> Charles

> --

> remove "_removeme" to reply.
> -------------------------------------------------------------------
> Charles Christacopoulos, Secretary's Office, University of Dundee,
> Dundee DD1 4HN, (Scotland) United Kingdom.
> Tel: +44+(0)1382-344891. Fax: +44+(0)1382-201604.
> http://somis.ais.dundee.ac.uk/
> Scottish Search Maestro http://somis2.ais.dundee.ac.uk/
> Win 2000? Still happy using OS2 Warp and so do the HTTPs above :-)



Mon, 06 Dec 2004 00:03:59 GMT  
 OS/2 classic REXX, SysTempFileName

Quote:



> >>It seems pointless do a RENAME when you can create the correct filename
> >>in the first place.

> >I don't think you can really say that when you don't know the context.

> >I use a similar approach when I have a process scanning a directory for
> >'work'. It only picks up files with a certain form of name; those not
> >conforming to that are left alone, because they are still being created
> >or modified.

>  From the original msg, there was the line:-


> To me, it looks as though 'TmpFile' is a newly created file which is
> subsequently renamed thereby using and extra system call.

> There may be some other context, but that is what I suspect is happening.

> --
> John

Yes John your assumption is correct.  I have to look at the code ... as it is
not originally mine.  Briefly (as I've been up since 5.30 in the morning ...
damn conferences) a file with any extention MUST be created in a specific
directory ... once all work is done on the file  ... then file must be
renamed with a specific extention.  Then when its turn comes it will be
picked up by another program ... processed ... and will be deleted.

So I have
tmpfile = systempfilename etc \mydirectory\file.tmp

I cannot work from scratch on an FWD file as it can be picked up by the next
process.
TmpFile must also be random as 2 or more processes may have kicked in doing
the same job for different datasets (the whole thing is on the back of a web
server ... so I want processes to kick in whenever they happen in real time,
not batch mode).

I will pursue this to know if there is bug with systempfilename ... in theory
it should produce 99,999 (in practice I can't see it as it may take for ever
to find at random the last filename).

I will try it in the next couple of days.
==================

In effect the process is a bit flowed ... I am looking at producing around
20000 files max.  Let alone it is too much to try and produce at random, if
the files were produced sequentially with a filename along the lines:
Unique_process_number+counter.ext
if anything crashes I will be left with orderly files (the second process
takes them sequentially and then deletes them).

Further to the above it is known (I suffered from it for a few months) that
if two processes kick in and call systempfilename it can and does produce the
same filenames for both.  It is as if its randomness is not random enough.
(With Search Maestro many times I ended up getting the URLs for one website
and I was then using them to retrieve files for another.  Tempfilenames were
getting mixed up).

--

remove "_removeme" to reply.
-------------------------------------------------------------------
Charles Christacopoulos, Secretary's Office, University of Dundee,
Dundee DD1 4HN, (Scotland) United Kingdom.
Tel: +44+(0)1382-344891. Fax: +44+(0)1382-201604.
http://somis.ais.dundee.ac.uk/
Scottish Search Maestro http://somis2.ais.dundee.ac.uk/
Win 2000? Still happy using OS2 Warp and so do the HTTPs above :-)



Mon, 06 Dec 2004 02:18:49 GMT  
 OS/2 classic REXX, SysTempFileName

   at 05:05 PM, Charles Christacopoulos

Quote:
>Can anyone take a guess as to what is causing me to get a sys1083. I have
>the following line in a rexx program:

>which results in sys1083.

How many processes are creating .fwd files?

Looking at the code I have for RexxUtil, it is not multi-process safe.

Quote:
>files similarly created.  There is a "known" problem in os/2 where it
>cannot cope with too many (how many?) files in one directory.  Could that

What known problem?  The WPS has limits, but that's not a file system
problem.  IAC, with 7K files, you are nowhere near any file system limit
that I know of.

Quote:
>I could increase the name of the file from 5 questionmarks to more by
>generating random letters, eg.  SysTempFileName('?????****.FWD') where *
>is a random letter.  Not sure what difference it might make.

You only need to do that if you can have 100K files backlogged.

Steven

--
-------------------------------------------------------------------

Warp4/FP15/14.085_W4 www.scoug.com irc.webbnet.org #scoug (Wed 7pm PST)
-------------------------------------------------------------------



Mon, 06 Dec 2004 04:01:17 GMT  
 OS/2 classic REXX, SysTempFileName

   at 07:18 PM, Charles Christacopoulos

Quote:
>in a specific directory ... once all work is done on the file  ... then
>file must be renamed with a specific extention.  Then when its turn comes
>it will be picked up by another program ... processed ... and will be
>deleted.

This all fine.  It's a unix-ish way of doing things and works well on unix
because creating and deleting small files has always been fast.

Quote:
>TmpFile must also be random as 2 or more processes may have kicked in
>doing the same job for different datasets (the whole thing is on the back
>of a web server ... so I want processes to kick in whenever they happen
>in real time, not batch mode).

I asked this before, but does this mean you might have two cgi's running
the rexx code at the same time?

Quote:
>I will pursue this to know if there is bug with systempfilename ... in
>theory it should produce 99,999 (in practice I can't see it as it may
>take for ever to find at random the last filename).
>Further to the above it is known (I suffered from it for a few months)
>that if two processes kick in and call systempfilename it can and does
>produce the same filenames for both.  It is as if its randomness is not

SysTempFileName has a very naive implementation.  Performance will degrade
in proportion to the number of files in the directory.  Also, there's no
guarantee that the file will not exist by the time you try to use the
name.  Your code must ensure this.  Based on your description of the
problem, wrapping the existing code in a retry loop might be sufficient.

Steven

--
-------------------------------------------------------------------

Warp4/FP15/14.085_W4 www.scoug.com irc.webbnet.org #scoug (Wed 7pm PST)
-------------------------------------------------------------------



Mon, 06 Dec 2004 04:01:39 GMT  
 OS/2 classic REXX, SysTempFileName


Quote:

>To me, it looks as though 'TmpFile' is a newly created file which is
>subsequently renamed thereby using and extra system call.

Nope.  SysTempFileName creates nothing.  However, it will issue a large
number of DosFindFirst system calls if the namespace is near full and you
are unlucky.  For some reason, IBM decided to limit the number of filter
characters to 5, so you can't make the odds better by enlarging the
namespace.

If performance is an issue,  I would use an alternative method of
calculating the unique file names.

Hint the code is in:

 \Toolkit\Samples\REXX\API\REXXUTIL\REXXUTIL.C

and anyone with eCS or MCP can look at it.

Steven

--
-------------------------------------------------------------------

Warp4/FP15/14.085_W4 www.scoug.com irc.webbnet.org #scoug (Wed 7pm PST)
-------------------------------------------------------------------



Mon, 06 Dec 2004 04:21:29 GMT  
 OS/2 classic REXX, SysTempFileName


Quote:
>I am not sure at all but since sys1083 means that "the" file (whatever
>file) either already exists, could not be found or is already in use, it
>might be a problem of the order of evaluating the REXX statement (who
>knows ?).

The code is syntactically correct. I vote for a race condition.  Unless, I
misread Charles post, the code does not fail every time.

Steven

--
-------------------------------------------------------------------

Warp4/FP15/14.085_W4 www.scoug.com irc.webbnet.org #scoug (Wed 7pm PST)
-------------------------------------------------------------------



Mon, 06 Dec 2004 04:25:42 GMT  
 
 [ 25 post ]  Go to page: [1] [2]

 Relevant Pages 

1. Rexx Macro Space on Classic OS/2 Rexx

2. Handling large file (classic rexx OS/2)

3. Sort function, classic rexx OS/2

4. os/2, classic rexx, and a pm interface

5. A SYS3175 error in classic os/2 rexx

6. OS/2 + Classic Rexx on a P4?

7. Difference in OS/2 Classic/Object REXX Read Only

8. how to get timezone in os/2 in classic rexx

9. Possible file open problem - OS/2 classic Rexx

10. Classic Rexx vs. Object Rexx

11. Classic REXX, Object REXX

12. Object REXX vs. Classic REXX

 

 
Powered by phpBB® Forum Software