DLL unload question for embedded Perl on Windows 
Author Message
 DLL unload question for embedded Perl on Windows

I ran into some problems when executing the sample code from perldoc
about embedding Perl in a C program. Here are my codes

---- embeddedperl.c begin ----
#include <EXTERN.h>               /* from the Perl distribution     */
#include <perl.h>                 /* from the Perl distribution     */

EXTERN_C void boot_DynaLoader (pTHX_ CV* cv);

EXTERN_C void xs_init(pTHX)
{
        char *file = __FILE__;
        /* DynaLoader is a special case */
        newXS("DynaLoader::boot_DynaLoader", boot_DynaLoader, file);

Quote:
}

void runperl()
{
        int ARGC = 2;
        char *ARGV[]={"","test.pl"};
        PerlInterpreter *my_perl;

        PERL_SYS_INIT3(&ARGC,(char ***)&ARGV,NULL);
        my_perl = perl_alloc();
        perl_construct(my_perl);
        PL_exit_flags |= PERL_EXIT_DESTRUCT_END;
        perl_parse(my_perl, xs_init, ARGC, ARGV, (char **)NULL);
        perl_run(my_perl);
        perl_destruct(my_perl);
        perl_free(my_perl);
        PERL_SYS_TERM();

Quote:
}

int main(int argc, char **argv, char **env)
{
        int i=0;
        for (i=0;i<2;i++)
             runperl();
Quote:
}

---- embeddedperl.c end ----

---- test.pl begin ----
use Cwd;

print cwd,"\n";
---- test.pl end ----

Here are the problems I got during execution

1. The loaded DLLs do not unload after the Perl interpreter is
shutdown
    In my example, after perl_parse() Cwd.dll will be loaded. I
expected this dll will be unloaded after calling
    perl_run() but it was not. How do I force all DLLs to be unloaded
after a script finishes?
2. perl_destruct() always throws exception. I have to comment out it
for my program to run. I suspect if it can run
    without problem, maybe my 1st question can be solved.
3. After commenting out perl_destruct(), my program throws exception
after calling runperl() the 2nd time. It
    actually died on perl_parse(). Since I wrap everything in a sub-
routine runperl() I thought everything starts from
    scratch. However It seems not. I have no idea how come it is OK
the 1st time but not the 2nd.

Hope somebody can shed light on these. Thanks very much.



Tue, 15 May 2012 11:46:30 GMT  
 DLL unload question for embedded Perl on Windows


Quote:
> I ran into some problems when executing the sample code from perldoc
> about embedding Perl in a C program. Here are my codes

> ---- embeddedperl.c begin ----
> #include <EXTERN.h>               /* from the Perl distribution     */
> #include <perl.h>                 /* from the Perl distribution     */

> EXTERN_C void boot_DynaLoader (pTHX_ CV* cv);

> EXTERN_C void xs_init(pTHX)
> {
>         char *file = __FILE__;
>         /* DynaLoader is a special case */
>         newXS("DynaLoader::boot_DynaLoader", boot_DynaLoader, file);
> }

> void runperl()
> {
>    int ARGC = 2;
>    char *ARGV[]={"","test.pl"};
>    PerlInterpreter *my_perl;

>    PERL_SYS_INIT3(&ARGC,(char ***)&ARGV,NULL);

This is wrong. The PERL_SYS macros must be called once each only (per
process), and must be called with the actual argv and env that were
passed to main. You can pass a different argv/env to perl_parse if you
need to.

- Show quoted text -

Quote:
>    my_perl = perl_alloc();
>    perl_construct(my_perl);
>    PL_exit_flags |= PERL_EXIT_DESTRUCT_END;
>    perl_parse(my_perl, xs_init, ARGC, ARGV, (char **)NULL);
>    perl_run(my_perl);
>    perl_destruct(my_perl);
>    perl_free(my_perl);
>    PERL_SYS_TERM();

> }

> int main(int argc, char **argv, char **env)
> {
>    int i=0;
>    for (i=0;i<2;i++)
>         runperl();
> }
> ---- embeddedperl.c end ----

> ---- test.pl begin ----
> use Cwd;

> print cwd,"\n";
> ---- test.pl end ----

> Here are the problems I got during execution

> 1. The loaded DLLs do not unload after the Perl interpreter is
> shutdown
>     In my example, after perl_parse() Cwd.dll will be loaded. I
> expected this dll will be unloaded after calling
>     perl_run() but it was not. How do I force all DLLs to be unloaded
> after a script finishes?

I would not expect Cwd.dll to be unloaded until after perl_free is
called. It is not normal for a perl interpreter to ever unload a loaded
extension dll.

Quote:
> 2. perl_destruct() always throws exception. I have to comment out it
> for my program to run. I suspect if it can run
>     without problem, maybe my 1st question can be solved.

I suspect this may have something to do with your misuse of SYS_INIT3
and SYS_TERM, but I don't know. If fixing that doesn't help, build perl
with -DDEBUGGING and see if you get more information.

Quote:
> 3. After commenting out perl_destruct(), my program throws exception
> after calling runperl() the 2nd time. It
>     actually died on perl_parse(). Since I wrap everything in a sub-
> routine runperl() I thought everything starts from
>     scratch. However It seems not. I have no idea how come it is OK
> the 1st time but not the 2nd.

This is expected. Your perl is built with threads (since you're on
WIn32) and you are effectively trying to use two different interpreters
from the same thread. Since you are not using the PERL_SET_CONTEXT
macros, this isn't going to work. Once you get perl_destruct working, I
suspect this will go away.

Ben



Tue, 15 May 2012 21:37:29 GMT  
 DLL unload question for embedded Perl on Windows

Quote:

> > 1. The loaded DLLs do not unload after the Perl interpreter is
> > shutdown

> I would not expect Cwd.dll to be unloaded until after perl_free is
> called. It is not normal for a perl interpreter to ever unload a loaded
> extension dll.

I just want to have a clean environment like a new process starts. Is
there any method to achieve this?

Quote:
> > 2. perl_destruct() always throws exception. I have to comment out it

> I suspect this may have something to do with your misuse of SYS_INIT3
> and SYS_TERM, but I don't know. If fixing that doesn't help, build perl
> with -DDEBUGGING and see if you get more information.

I modified my code in this way

int main(int argc, char **argv, char **env)
{
        int i=0;
        PERL_SYS_INIT3(&argc,&argv,NULL);
        for (i=0;i<2;i++)
             runperl();
        PERL_SYS_TERM();

Quote:
}

and the result is a bit confusing. It ran fine on the machine that
build the executable but still threw exception on perl_destruct() on
another machine. The only difference I can tell is one machine has MS
Visual Studio 2005 and the other not.

Quote:

> This is expected. Your perl is built with threads (since you're on
> WIn32) and you are effectively trying to use two different interpreters

Multi-thread is another problem I met but didn't mention. I'm using
Active Perl 5.8.8 but my program always crash with multi-thread.
Currently I just remove the thread functions and want to solve the
problems I mentioned first.


Wed, 16 May 2012 09:27:00 GMT  
 DLL unload question for embedded Perl on Windows


Quote:


> > > 1. The loaded DLLs do not unload after the Perl interpreter is
> > > shutdown

> > I would not expect Cwd.dll to be unloaded until after perl_free is
> > called. It is not normal for a perl interpreter to ever unload a loaded
> > extension dll.

> I just want to have a clean environment like a new process starts. Is
> there any method to achieve this?

I don't know. Have you checked to see whether perl_free unloads the dlls
or not? If it does then that's what you want.

Quote:
> > > 2. perl_destruct() always throws exception. I have to comment out it

> > I suspect this may have something to do with your misuse of SYS_INIT3
> > and SYS_TERM, but I don't know. If fixing that doesn't help, build perl
> > with -DDEBUGGING and see if you get more information.

> I modified my code in this way

> int main(int argc, char **argv, char **env)
> {
>         int i=0;
>         PERL_SYS_INIT3(&argc,&argv,NULL);

Um, what did I say? *You need to pass the same arguments as were passed
to main*. That *includes* env.

Quote:
>         for (i=0;i<2;i++)
>              runperl();
>         PERL_SYS_TERM();
> }

> and the result is a bit confusing. It ran fine on the machine that
> build the executable but still threw exception on perl_destruct() on
> another machine. The only difference I can tell is one machine has MS
> Visual Studio 2005 and the other not.

As a general rule you must use the same compiler your copy of perl was
built with. If you are using 32-bit AS perl that means MSVC 6. If you
don't have the right compiler, rebuild perl with the compiler you *do*
have.

Quote:
> > This is expected. Your perl is built with threads (since you're on
> > WIn32) and you are effectively trying to use two different interpreters

> Multi-thread is another problem I met but didn't mention. I'm using
> Active Perl 5.8.8 but my program always crash with multi-thread.
> Currently I just remove the thread functions and want to solve the
> problems I mentioned first.

Have you read *all* of perlembed? It explains wuite carefully what you
must do to handle multiple threads.

Ben



Wed, 16 May 2012 18:53:27 GMT  
 DLL unload question for embedded Perl on Windows

Quote:
> I don't know. Have you checked to see whether perl_free unloads the dlls
> or not? If it does then that's what you want.

AFAIK, the design of Perl DLLs was always that they are not unloadable.

Hope this helps,
Ilya



Thu, 17 May 2012 02:10:42 GMT  
 DLL unload question for embedded Perl on Windows


Quote:

> > I don't know. Have you checked to see whether perl_free unloads the dlls
> > or not? If it does then that's what you want.

> AFAIK, the design of Perl DLLs was always that they are not unloadable.

That's more-or-less what I thought, but perldoc DynaLoader says

|    dl_unload_file()
|        Syntax:
|
|            $status = dl_unload_file($libref)
|
|        Dynamically unload $libref, which must be an opaque library
|        reference as returned from dl_load_file.  Returns one on success
|        and zero on failure.
|
|        This function is optional and may not necessarily be provided on
|        all platforms.  If it is defined, it is called automatically when
|        the interpreter exits for every shared object or library loaded by
|        DynaLoader::bootstrap.  All such library references are stored in

|        The files are unloaded in lastin, firstout order.
|
|        This unloading is usually necessary when embedding a sharedobject
|        perl (e.g.  one configured with ?Duseshrplib) within a larger
|        application, and the perl interpreter is created and destroyed
|        several times within the lifetime of the application.  In this case
         <details of why this is necessary>

and checking dl_win32.xs shows that it does define dl_unload_file.
Therefore, I would expect all perl extension dlls to be unloaded as part
of perl_destruct, but not before.

Ben



Thu, 17 May 2012 02:42:35 GMT  
 DLL unload question for embedded Perl on Windows

Quote:
>> AFAIK, the design of Perl DLLs was always that they are not unloadable.
> That's more-or-less what I thought, but perldoc DynaLoader says
>|    dl_unload_file()
>|        Syntax:
>|
>|            $status = dl_unload_file($libref)
>|
>|        Dynamically unload $libref, which must be an opaque library
>|        reference as returned from dl_load_file.  Returns one on success
>|        and zero on failure.

This is just an interface to the OS's implementation; it has practically
nothing to do with Perl's DLLs.

One may keep in mind a simple analogy: in C, one can free(); but if
you have an array of (pointers to) structures, it is not enough to
free() the arena where the array content is situated.  One must know
the semantic of array elements, and do recursive free()ing (or
decrement of refcounts; or whatever is needed).

dl_unload_file() is just a dumb free().  It knows nothing about what
one should do with dangling pointers inside the arena, or pointers
which were stored in the area.

Hope this helps,
Ilya



Thu, 17 May 2012 09:38:25 GMT  
 DLL unload question for embedded Perl on Windows


Quote:

> >> AFAIK, the design of Perl DLLs was always that they are not unloadable.

> > That's more-or-less what I thought, but perldoc DynaLoader says

> >|    dl_unload_file()
> >|        Syntax:
> >|
> >|            $status = dl_unload_file($libref)
> >|
> >|        Dynamically unload $libref, which must be an opaque library
> >|        reference as returned from dl_load_file.  Returns one on success
> >|        and zero on failure.

> This is just an interface to the OS's implementation; it has practically
> nothing to do with Perl's DLLs.

You snipped the important bit. DynaLoader will in fact call
dl_unload_file (if defined) on all previously-loaded extension dlls
during global destruction.

Quote:
> One may keep in mind a simple analogy: in C, one can free(); but if
> you have an array of (pointers to) structures, it is not enough to
> free() the arena where the array content is situated.  One must know
> the semantic of array elements, and do recursive free()ing (or
> decrement of refcounts; or whatever is needed).

> dl_unload_file() is just a dumb free().  It knows nothing about what
> one should do with dangling pointers inside the arena, or pointers
> which were stored in the area.

Quite so, which is why this can't (easily) be done *before* global
destruction.

Ben



Thu, 17 May 2012 11:07:48 GMT  
 DLL unload question for embedded Perl on Windows

Quote:
>> This is just an interface to the OS's implementation; it has practically
>> nothing to do with Perl's DLLs.

> You snipped the important bit. DynaLoader will in fact call
> dl_unload_file (if defined) on all previously-loaded extension dlls
> during global destruction.

This must be a somewhat new development; if so, I have no idea why
this is supposed to work (definitely, in my Perl DLLs, I have no
provision for unloading...).

Quote:
> Quite so, which is why this can't (easily) be done *before* global
> destruction.

... and, AFAIU, *after* global destruction too.  Perl DLLs have BOOT:
sections; they do not have UNBOOT: ones to release resources they
allocated.

Ilya



Thu, 17 May 2012 19:59:03 GMT  
 DLL unload question for embedded Perl on Windows


Quote:

> >> This is just an interface to the OS's implementation; it has practically
> >> nothing to do with Perl's DLLs.

> > You snipped the important bit. DynaLoader will in fact call
> > dl_unload_file (if defined) on all previously-loaded extension dlls
> > during global destruction.

> This must be a somewhat new development; if so, I have no idea why
> this is supposed to work (definitely, in my Perl DLLs, I have no
> provision for unloading...).

OK... checking the source, it appears the functionality was added in
2000, but disabled (by default) in 2001. So, you are correct that perl
does not in fact currently ever unload extension DLLs. (I wonder what
happened to the original bug this was supposed to solve, that of
unloading and reloading libperl.so causing all loaded extensions to hold
pointers into an unloaded library?)

Ben



Thu, 17 May 2012 21:47:48 GMT  
 DLL unload question for embedded Perl on Windows
I got my problems fixed by building my own Perl even though I prefer
to use ActivePerl. Anyway it works for now. Since I need to use a DLL
to run the Perl interpreter, I don't have the arg/argv/env arguments.
However it still works with NULL passed. Not sure if there's any
potential risks. It works fine so far. As for unloading DLL problem, I
add a compiler flag DL_UNLOAD_ALL_AT_EXIT and Perl interpreter will
unload all DLLs after perl_destruct or perl_free (sorry I forgot it).
Thanks for all the valuable information. For a layman of Perl guts,
all the information mentioned help me a lot.


Mon, 21 May 2012 12:07:31 GMT  
 DLL unload question for embedded Perl on Windows


Quote:
> I got my problems fixed by building my own Perl even though I prefer
> to use ActivePerl. Anyway it works for now. Since I need to use a DLL
> to run the Perl interpreter, I don't have the arg/argv/env arguments.
> However it still works with NULL passed. Not sure if there's any
> potential risks. It works fine so far. As for unloading DLL problem, I
> add a compiler flag DL_UNLOAD_ALL_AT_EXIT and Perl interpreter will
> unload all DLLs after perl_destruct or perl_free (sorry I forgot it).
> Thanks for all the valuable information. For a layman of Perl guts,
> all the information mentioned help me a lot.

Ilya is right, though, that perl extensions are not designed to be
unloaded (since DL_UNLOAD_ALL isn't normally defined). You are likely to
get memory and other resource leaks from unloading and reloading
extensions that statically allocate resources.

Ben



Mon, 21 May 2012 14:51:50 GMT  
 DLL unload question for embedded Perl on Windows

Quote:

> Ilya is right, though, that perl extensions are not designed to be
> unloaded (since DL_UNLOAD_ALL isn't normally defined). You are likely to
> get memory and other resource leaks from unloading and reloading
> extensions that statically allocate resources.

> Ben

So your suggestion is not to enable this flag? If so, will it make any
difference in the behavior of the same script executed several times?
I am asking because my script hanged when running the 2nd time. But it
happened when I used ActivePerl. Maybe I will recompile Perl again
without this flag if I got any resource leak issue.


Tue, 22 May 2012 02:17:56 GMT  
 DLL unload question for embedded Perl on Windows


Quote:

> > Ilya is right, though, that perl extensions are not designed to be
> > unloaded (since DL_UNLOAD_ALL isn't normally defined). You are likely to
> > get memory and other resource leaks from unloading and reloading
> > extensions that statically allocate resources.

> So your suggestion is not to enable this flag? If so, will it make any
> difference in the behavior of the same script executed several times?
> I am asking because my script hanged when running the 2nd time. But it
> happened when I used ActivePerl. Maybe I will recompile Perl again
> without this flag if I got any resource leak issue.

I'm afraid I don't know. If you can ensure your DLL won't itself be
unloaded and reloaded then omitting the flag is probably the right
answer, assuming I've understood everyhing correctly. If you can't then
there may not be a solution, short of going through any XS modules you
might use and auditing them for potential resource leaks. Modules are
generally written for and tested on perl(1), so getting too far away
from the way perl(1) uses the perl DLL may prove difficult.


are likely to be people there who understand this better than I do.

Ben



Tue, 22 May 2012 09:35:04 GMT  
 
 [ 14 post ] 

 Relevant Pages 

1. Q:Newbie-Howto: Embedded WIN32-Perl in a DLL

2. Help: Embedded Perl 5.002 Port DLL fails on Win95

3. Embedding Perl in a DLL

4. ANSWER:embedding Perl interpreter in *DLL*

5. embedding Perl in *DLL*?

6. Perl on Windows - use Perl as DLL

7. Building Perl on Windows as Static Library (as opposed to a DLL)

8. MSDOS/Windows Perl & Winsock.DLL

9. Perl's goodies in a Windows DLL

10. Building Perl on Windows as Static Library (as opposed to a DLL)

11. windows nt user32.dll initialization and perl

12. Windows DLL support for perl

 

 
Powered by phpBB® Forum Software