Perl 5.6 regexp error message: Bug? 
Author Message
 Perl 5.6 regexp error message: Bug?

Fellow Perl-ers,

I installed perl 5.6 on a freeBSD box, and discovered that a script of mine
gives the following error:

Complex regular subexpression recursion limit (32766) exceeded at ...

The thing I'm trying to match on is very simple:
        $a =~ /Permission denied/;
but $a is very long, ~ 300K characters.

What is recursing here, and why is there a limit on the length of the input
string?  I thought that "no arbitrary limits" was one of the many
advantages of perl.

Rob
--
Dr. Robert K. Cunningham            Information System Technology Group

*** My comments, my opinions: my responsibility.
PGP key available from http://www.*-*-*.com/ :11371



Mon, 28 Oct 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?
[A complimentary Cc of this posting was sent to Robert Cunningham


Quote:
> Complex regular subexpression recursion limit (32766) exceeded at ...

> The thing I'm trying to match on is very simple:
>    $a =~ /Permission denied/;
> but $a is very long, ~ 300K characters.

Cannot happen.

Hope this helps,
Ilya



Tue, 29 Oct 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?
What cannot happen?  
        (1) You cannot fix this.
        (2) My bug report is incorrect, in your estimation.  [It is not.]
        (3) Perl cannot handle long variables.
        (4) Something else.
Please explain.

More importantly, how do I work around this perl regexp bug?

Rob

Quote:

> [A complimentary Cc of this posting was sent to Robert Cunningham


> > Complex regular subexpression recursion limit (32766) exceeded at ...

> > The thing I'm trying to match on is very simple:
> >       $a =~ /Permission denied/;
> > but $a is very long, ~ 300K characters.

> Cannot happen.

> Hope this helps,
> Ilya

--
Dr. Robert K. Cunningham            Information System Technology Group

*** My comments, my opinions: my responsibility.
PGP key available from http://pgpkeys.mit.edu:11371


Tue, 29 Oct 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?

Quote:

> Complex regular subexpression recursion limit (32766) exceeded at ...

> The thing I'm trying to match on is very simple:
>    $a =~ /Permission denied/;
> but $a is very long, ~ 300K characters.

Here's something to try: Make a small program which does nearly nothing
but cause this error. That is, first it would read the very long string
from a file. (You may need to hack your original program to write that
file, of course.) The small program would then do the pattern match and
generate the error message. Can you get that to work, and post the small
program here?

That will help to rule out some unlikely possibilities, such as that
you're running another program which is generating that error, or that
it's a different pattern causing the problem than the one that you think
it is.

Cheers!

--
Tom Phoenix       Perl Training and Hacking       Esperanto
Randal Schwartz Case:     http://www.rahul.net/jeffrey/ovs/



Tue, 29 Oct 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?

Quote:

>I installed perl 5.6 on a freeBSD box, and discovered that a script of mine
>gives the following error:

>Complex regular subexpression recursion limit (32766) exceeded at ...

All of the messages that Perl might issue are documented
in perldiag.pod.

For your message de jure it says:

----------------------
=item Complex regular subexpression recursion limit (%d) exceeded

(W regexp) The regular expression engine uses recursion in complex situations
where back-tracking is required.  Recursion depth is limited to 32766,
or perhaps less in architectures where the stack cannot grow
arbitrarily.  ("Simple" and "medium" situations are handled without
recursion and are not subject to a limit.)  Try shortening the string
under examination; looping in Perl code (e.g. with C<while>) rather
than in the regular expression engine; or rewriting the regular
expression so that it is simpler or backtracks less.  (See L<perlbook>
for information on I<Mastering Regular Expressions>.)
----------------------

Surely your pattern has got to be "Simple", so maybe you
have found a bug, but I cannot reproduce it.

Quote:
>The thing I'm trying to match on is very simple:
>    $a =~ /Permission denied/;

But it can't really look like that (with the semicolon
separator) in your real code.

Right?

[ because a match like that is useless if it is not in a conditional...  
  errr, unless you want to undef all of the regex memories or something.
]

If you just want to get past the problem, you can use index()
to search for constant strings instead of a pattern match.

Quote:
>but $a is very long, ~ 300K characters.

Works fine for me under 5.6 with a 2Mb file:

-------------
#!/usr/bin/perl -w
use strict;

local $/;
my $s = <>;  # slurp entire file

if ( $s =~ /Permission denied/ ) {
   print "matched\n";

Quote:
}

-------------

Using $a or $b for non-sorting purposes gives me a sense
of foreboding, so I changed the variable name  :-)

Quote:
>What is recursing here,

Sounds like a bug. Your pattern does not need any backtracking.

Quote:
>and why is there a limit on the length of the input
>string?  I thought that "no arbitrary limits" was one of the many

                          ^^

Quote:
>advantages of perl.

That's a bit too strong.

More like "as few arbitrary limits as possible".

Variable names are limited to 255 characters (run away screaming
if you begin to hit that limit :-)

--
    Tad McClellan                          SGML Consulting

    Fort Worth, Texas



Tue, 29 Oct 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?

Quote:

> >       $a =~ /Permission denied/;

> But it can't really look like that (with the semicolon
> separator) in your real code.  Right?
> [ because a match like that is useless if it is not in a conditional...  

Maybe he's using ( $`, $&, $' ).

--
John Porter



Tue, 29 Oct 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?

Quote:

>What cannot happen?  
>    (1) You cannot fix this.
>    (2) My bug report is incorrect, in your estimation.  [It is not.]
>    (3) Perl cannot handle long variables.
>    (4) Something else.
>Please explain.

To be fair, it is a rather incredible bug report; it's simply
unbelievable that a "subexpression recursion" error would be
triggered by that pattern (but I promise to try it out as soon
as possible). There is very probably some context that's missing
from your description. Could you give more detail?
Quote:
>> > Complex regular subexpression recursion limit (32766) exceeded at ...

>> > The thing I'm trying to match on is very simple:
>> >       $a =~ /Permission denied/;
>> > but $a is very long, ~ 300K characters.



Tue, 29 Oct 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?

Quote:


>> >   $a =~ /Permission denied/;

>> But it can't really look like that (with the semicolon
>> separator) in your real code.  Right?
>> [ because a match like that is useless if it is not in a conditional...  

>Maybe he's using ( $`, $&, $' ).

It is still useless (or at least "dangerous").

Those are only set for successful matches, so if you want to
_rely_ on them, you need to see if the match succeeded
before you use them...

... so you _still_ need a conditional check.

Still anxiously awaiting where a bare match (sans parens, sans m//g)
can be (safely) useful outside of a conditional.

:-)

--
    Tad McClellan                          SGML Consulting

    Fort Worth, Texas



Tue, 29 Oct 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?


:>> >  $a =~ /Permission denied/;
:>>
:>> But it can't really look like that (with the semicolon
:>> separator) in your real code.  Right?
:>> [ because a match like that is useless if it is not in a conditional...
:>
:>Maybe he's using ( $`, $&, $' ).
:
:It is still useless (or at least "dangerous").
:
:Those are only set for successful matches, so if you want to
:_rely_ on them, you need to see if the match succeeded
:before you use them...
:
:... so you _still_ need a conditional check.

What about when the match is guaranteed to always succeed?

:Still anxiously awaiting where a bare match (sans parens, sans m//g)
:can be (safely) useful outside of a conditional.

Perhaps:
  $string =~ /A*/;
  print "begins with ", length($&), " As\n";

-or-

  eval { '' =~ /$pattern/ };

--
Py (Amateur Radio: KF6WFP) -- 3.141592653589793238462643383...
Pythagoras Watson -- "Live long and may all your kernels pop."



Wed, 30 Oct 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?

Quote:

>Still anxiously awaiting where a bare match (sans parens, sans m//g)
>can be (safely) useful outside of a conditional.

As the last statement of a subroutine, so the sub returns the
result of the match?

  -Greg
--

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::



Thu, 31 Oct 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?
Alas, the bug does not occur a long string is read into a stand along
program and matched against the same simple regexp. However, if the string
is processed in the midst of my large application, perl dies an untimely
death, giving the complex regular subexpression recursion error.  My guess
is that there is some leftover state between calls in the regexp engine.  I
tested this hypothesis by enclosing the regexp search in a for loop,
repeating the test 10000 times--but that didn't cause the error either, so
it's a more complicated interaction, perhaps between different types of
regexp calls.  For the record, I have a great many more complicated
regexps, including forward and backward negative assertions, [] and ()
options, etc.

Can anyone suggest an alternative test to try, short of attempting to pull
apart the entire application?  Other than $', $& or $`, (which I do not
use) what are the other stateful components of the regexp engine?

Rob
--
Dr. Robert K. Cunningham            Information System Technology Group

*** My comments, my opinions: my responsibility.
PGP key available from http://pgpkeys.mit.edu:11371



Sat, 02 Nov 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?
: Alas, the bug does not occur a long string is read into a stand along
: program and matched against the same simple regexp. However, if the string

: Can anyone suggest an alternative test to try, short of attempting to pull

not I, but don't over look the possibility that the real bug is the error
message.  Perhaps something else is going wrong and the message is
misleading.  



Sat, 02 Nov 2002 03:00:00 GMT  
 Perl 5.6 regexp error message: Bug?

Quote:

> Can anyone suggest an alternative test to try, short of attempting to pull
> apart the entire application?  Other than $', $& or $`, (which I do not
> use) what are the other stateful components of the regexp engine?

Are you *absolutely certain* that this simple regular expression is the
one giving the warning?  This seems like the most likely answer here.

See perldebug/"Debugging regular expresions", which will direct you
to perldebguts and use re 'debug' (and be prepared for scads of output
from the latter).



Sat, 02 Nov 2002 03:00:00 GMT  
 
 [ 13 post ] 

 Relevant Pages 

1. bincancel:11 large binaries:AR592:@@NCM

2. perl 5.6 error installing Dave Roths Admin tools, e.g win32::message

3. Perl 5.6, eval, and Illegal octal digit message

4. DELPHI/ CAREER POSITION/ IN

5. Perl 5.6, Regexp, performance problems

6. Perl 5.6 says 'syntax error', Perl 5.005_03 does not

7. Threads bug in Perl 5.6??

8. Perl 5.6 on Linux bug?

9. Threads bug in Perl 5.6??

10. perl 5.6: bug with -0777 slurping?

11. Perl 5.6 bug?

12. Perl 5.6 Bug: rmdir

 

 
Powered by phpBB® Forum Software