Tough one: how to chain ties? 
Author Message
 Tough one: how to chain ties?

Oh ye Perl Gurus,

I'm trying to write a module for tie and want to chain in
another module. Both modules implement hashes, so this
should be possible in principle.

To be more concrete: I want to write a module that gives
persistency to a hash, i.e. it stores the hash data into
a file upon `untie' and restores them upon `tie'. It would
be nice to be able to use another `tie' package (IxHash, to
be specific) for the internal representation of that hash.
This is what I call `chaining'.

Example code:

 tie %a, Tie::IxHash;
 tie %b, Tie::PersistentHash, $filename, \%a;

 # now work with %b, which is persistent AND order-preserving

 untie %b; # stores the data to file

OK, I can dump out the IxHash structure via Data::Dumper and read it
back in via eval. But I cannot re-tie the dumped data to
the internal PersistentHash hash var. I can copy the data back
into a newly tied hash, but I would lose (in the IxHash case)
the order of the keys.

What I have this far (excluding error checking etc.):

sub TIEHASH {
  my $self = [];
  $self->[1] = shift; # filename
  $self->[0] = shift || {}; # chained tied hashref (optional)

  do $self->[1] if -f $self->[1]; # read dumped data
  $self->[0] = $PersistentHash;  # XXX here is the problem...

Quote:
}

sub FETCH    { $_[0]->[0]{$_[1]} }
sub STORE    { $_[0]->[0]{$_[1]} = $_[2] }
sub FIRSTKEY { my $a = scalar keys %{$_[0]->[0]}; each %{$_[0]->[0]} }
sub NEXTKEY  { each %{$_[0]->[0]} }
sub EXISTS   { exists $_[0]->[0]->{$_[1]} }
sub DELETE   { delete $_[0]->[0]->{$_[1]} }
sub CLEAR    { %{$_[0]->[0]} = () }

sub DESTROY {
  my $self = shift;

  use Data::Dumper;
  open DB, ">$self->[1]";
  if (tied $self->[0]) {
    print DB Data::Dumper->Dump([tied %{$self->[0]}], [qw(PersistentHash)]);
  } else {
    print DB Data::Dumper->Dump([$self->[0]], [qw(PersistentHash)]);
  }
  close DB;
  # dumped data file looks like:
  # $PersistentHash = bless({
  #   'key' => 'data',
  #   ...
  # }, 'Tie::IxHash');

Quote:
}

The line with the XXX is the problem: here I would like to
re-tie the dumped data struct to the internal hash ($self->[0]).

The problem is I cannot find a way to get the object data of
the IxHash back into the tied variable.

Naively, I tried

 my %a;
 tie %a, $PersistentHash;
 $self->[0] = \%a;

which would give an error:

 Can't locate object method "FIRSTKEY" via package "Tie::IxHash=ARRAY(0x10ce48)"

Then I tried

 tie %a, ref($PersistentHash);

which ties the variable to the right package, but the hash is empty.

--- Now how can I move those data back into the tied object? ---

It would be nice if

 tie %a, $xref;

would do the right thing, i.e. tie the variable to the module
$xref is blessed into AND initialize it with the contents of $xref.

Perl Gods, maybe in v5.006?

Any suggestions?

Roland
--

ALCATEL Austria, Dept. RTPM       FAX:   +43-1-27722-3955
Scheydgasse 41, A-1210 WIEN, Austria (no Kangaroos here!)



Tue, 13 Nov 2001 03:00:00 GMT  
 Tough one: how to chain ties?

Quote:
> I did do this, and it was very easy, and it almost worked.  The reason
> it did not work is because I used DB_File $DB_RECNO as the persistent
> array class, and it has a bug and dumps core.  

It turns out that the bug is fixed in DB_File 1.65, and it *does* work.


Tue, 13 Nov 2001 03:00:00 GMT  
 Tough one: how to chain ties?

Quote:
>  tie %a, Tie::IxHash;
>  tie %b, Tie::PersistentHash, $filename, \%a;

Oh, this is an interesting approach.

Quote:
> sub TIEHASH {
>   my $self = [];
>   $self->[1] = shift; # filename
>   $self->[0] = shift || {}; # chained tied hashref (optional)

>   do $self->[1] if -f $self->[1]; # read dumped data
>   $self->[0] = $PersistentHash;  # XXX here is the problem...
> }

The first thing I notice here is that you have the arguments to
TIEHASH wrong.  The first argument will be the class name, not the
filename.  So you needed to have

Quote:
> sub TIEHASH {
>   my $self = [];

    my $class = shift;
Quote:
>   $self->[1] = shift; # filename
>   $self->[0] = shift || {}; # chained tied hashref (optional)

>   do $self->[1] if -f $self->[1]; # read dumped data
>   $self->[0] = $PersistentHash;  # XXX here is the problem...

    bless $self => $class;

Quote:
> }

I guess you got this right in your implementation and forgot to
include it only in the news article?  Because actually I think it
needs to look more like this:

Quote:
> sub TIEHASH {
>   my $self = [];

    my $class = shift;
Quote:
>   $self->[1] = shift; # filename
>   $self->[0] = shift || {}; # chained tied hashref (optional)

    if (-f $self->[1]) {
      do $self->[1];
Quote:
>     $self->[0] = $PersistentHash;  # XXX here is the problem...

    }
    bless $self => $class;

Quote:
> }

But OK, your real problem is that you can read in the Tie::IXHash
object that you saved, and you can reinstate it as a Tie::IxHash
object, but you can't reassociate it with the %a hash.

There are a couple of ways around it.  One way is that the object
doesn't actually have to be tied to anything to function as an
interface to a tied hash class.  Forget about %a completely, and just
use the IxHash object directly.  Then to set it up would look like
this:

        my $a = (tie {} => Tie::IxHash);
        tie %b => Tie::PersistentHash, $filename, $a;

Now instead of this:

Quote:
> sub FETCH    { $_[0]->[0]{$_[1]} }

use this:

  sub FETCH    { $_[0]->[0]->FETCH($_[1]) }

and instead of this:

Quote:
> sub STORE    { $_[0]->[0]{$_[1]} = $_[2] }

use this:

Quote:
> sub STORE    { $_[0]->[0]->STORE($_[1], $_[2]) }

This is probably the easiest thing to do.  In this case your TIEHASH
method will look like this:

Quote:
> sub TIEHASH {
>   my $self = [];

    my $class = shift;
Quote:
>   $self->[1] = shift; # filename

    if (-f $self->[1]) {
      do $self->[1];
Quote:
>     $self->[0] = $PersistentHash;  # XXX no problem any more!

    } else {
      $self->[0] = Tie::IxHash->TIEHASH()  # LOD!
    }
    bless $self => $class;

Quote:
> }

That is a lot to absorb, and I think it will solve your problem, so
you might want to put this message aside now and come back to the rest
of it later if you are still interested.

Another, totally different strategy is to make a class just like
Tie::IxHash, except that instead of creating a new object from
scratch, it will let you specify and old object as an initializer.
Then you can read the old object out of the file with Data::Dumper and
hand it off to the initializer for the new class.

You would do that like this:

        package My::IxHash;
        use Tie::IxHash;

        sub TIEHASH {
          my $package = shift;
          my $old_object = shift;
          # It's already constructed, so just return it!
          $old_object;          # LOD!
        }

        1;

Now the TIEHASH method for Tie::PersistentHash will look like this:

Quote:
> sub TIEHASH {
>   my $self = [];

    my $class = shift;
Quote:
>   $self->[1] = shift; # filename

    if (-f $self->[1]) {
      do $self->[1];
      my %inner_hash;
      # Attach inner hash to old Tie::IxHash object
      tie %inner_hash => My::IxHash, $PersistentHash;
      $self->[0] = \%inner_hash;

      $self->[0] = shift;    # Use tied hash (\%a) supplied in `tie' call
    } else {
      my %inner_hash;
      tie %inner_hash => Tie::IxHash; # Create a new tied hash
      $self->[0] = \%inner_hash;      # and use it
    }
    bless $self => $class;

Quote:
> }
> It would be nice if

>  tie %a, $xref;

> would do the right thing, i.e. tie the variable to the module
> $xref is blessed into AND initialize it with the contents of $xref.

That's a nice idea, and I can't see anything wrong with it, except
that it might be difficult to explain what it was for.

Quote:
> Perl Gods, maybe in v5.006?

Patches welcome.

Quote:
> Any suggestions?

Hope this helps.

################################################################

Before I had read the details of what you wanted, I wrote a long
article about a totally different way to do it, which may be useful,
or at least intersting, so it follows here:

Quote:
> I'm trying to write a module for tie and want to chain in
> another module. Both modules implement hashes, so this
> should be possible in principle.
> To be more concrete: I want to write a module that gives
> persistency to a hash, i.e. it stores the hash data into
> a file upon `untie' and restores them upon `tie'. It would
> be nice to be able to use another `tie' package (IxHash, to
> be specific) for the internal representation of that hash.
> This is what I call `chaining'.

That seems backwards to me.  You have two sets of features you want to
combine:

        1. Persistence  (Let's suppose that you want NBDM_File semantics.)
        2. Indexedness  (Let's suppose that you want Tie::IxHash.)

The `persistent' part is implemented with a lot of complicated C code
and there is a lot of complicated stuff going on under the hood.

The `indexedness' part is implemented in Perl using a bunch of Perl
data structures.  

You can either add indexedness to NDBM_File, or add persistence to
Tie::IxHash.  

One good way to add features to something in Perl is to use the `tie'
facility.  This adds an extra layer of semantics to a perl data
structure.  It makes sense to do this to Tie::IxHash because it is
implemented in perl with Perl data structures.  It is conceivable that
by magicking the data structures in Tie::IxHash, you could make them
persistent.  

But the data structures used to implement NDBM_File are *not* Perl
data structures; they are in C, and it is impossible to tie them.  So
it is probably hopeless to expect to make NDBM_File indexed.

So that is why when you said

Quote:
> It would be nice to be able to use another `tie' package (IxHash, to
> be specific) for the internal representation of that hash.

that it sounded backwards to me.  

To lift the argument up to a more abstract level: Persistence is not a
very big *interface* change.  The interface to a persistent hash is
exactly the same as the interface to any other hash.  

But indexedness is a very large interface change.  There all all sorts
of new operations that you have to support, such as store/fetch by
index, push, pop, and so on.

You care going to layer together three pieces of software:

        Your program
        ----------------------top interface
        Hash module 1
        ----------------------bottom interface
        Hash module 2

To minimize the amount of work you have to do, you want the interface
change to be at the top interface.  Otherwise, both hash modules will
have to know about it.  So you want to put Tie::IxHash on top:

        Your program
        ---------------------- interface for indexedness
        Tie::IxHash
        ---------------------- interface for persistence
        NDBM_File

This seems plausible, because the interface for persistence is going
to be exactly like the regular hash/array interface that Tie::IxHash
was going to use anyway.  

To do this, you need to arrange that the internal data structures that
Tie::IxHash uses become persistent.  This is not too hard.  The inside
of a Tie::IxHash object is a hash, a scalar, and two arrays.  If you
can find a persistent hash class and a persistent array classes, you
can tie the hash and the arrays and then the IxHash object will be
persistent.

I did do this, and it was very easy, and it almost worked.  The reson
it did not work is because I used DB_File $DB_RECNO as the persistent
array class, and it has a bug and dumps core.  Here is the code:

        package Persistent::IxHash;
        use Tie::IxHash;

        use DB_File;

        sub TIEHASH {
          my $c = shift;
          my $f = shift;
          my($s) = [];
          my $p = DB_File;




          $s->[0] = \%keyhash;   # hashkey index



          bless $s, $c;

          return $s;
        }

        # All other methods are inherited from Tie::IxHash

        1;

This TIEHASH method is just like the one in Tie::IxHash.  It
constructs and object $s which contains a hash and two arrays.  I took
the code from from Tie::IxHash and changed it to tie the hash and the
arrays before returning the object.  This object will inherit its
FETCH and STORE and other methods from Tie::IxHash, and Tie::IxHash
will look into the object and see the hash and the two arrays as it
should.  It does not need to know that they are persistent.

To use this, you say

        use Persistent::IxHash;
        use Fcntl;
        my $o = tie %h => Persistent::IxHash, $filename, $flags, $mode;

and then $o and %h behave just like a Tie::IxHash object.  Or they
would, except that DB_File dumps core.  But if you can find a
different persistent array class that is not broken it will work just
fine.  It would not be too hard to write one.  You could build it on
top of a persistent hash by using record numbers as keys.

################################################################

Other work in similar areas:

1. I just released Tie::HashHistory, which is a tied hash layer that
   you can insert between your own program and any other tied hash
   class.  Everything looks completely ordinary, but you can also ask
   HashHistory for the history of a key. It will return a list of all
   the values that the key has ever had, in order.

   http://www.plover.com/~mjd/perl/HashHistory/

2. At the Boston O'Reilly tutorials, Mike Cariaso ...

read more »



Tue, 13 Nov 2001 03:00:00 GMT  
 Tough one: how to chain ties?

There were various minor errors, so in hopes of clarifying, Here are a
couple of corrections:



Quote:
>    my $a = (tie {} => Tie::IxHash);
>    tie %b => Tie::PersistentHash, $filename, $a;

This won't work; you'd have to do something like

        my $a = do { my %a; tie %a => Tie::IxHash };
        tie %b => Tie::PersistentHash, $filename, $a;

The goal here being to use the `tie' on something you can get rid of
right away.  But actually the %a is irrelevant here, and what you're
really after is the associated object.  So if you take this first
approach, you might as well just call the constructor directly and say

        my $a = Tie::IxHash->TIEHASH();
        tie %b => Tie::PersistentHash, $filename, $a;

which is more in line with the way the other methods work anyway:

Quote:
>> sub STORE    { $_[0]->[0]->STORE($_[1], $_[2]) }

But then actually, although I didn't say so, if you use the
Tie::Persistent::TIEHASH method I showed, you don't need to bother
with $a at all; the Tie::PersistentHash::TIEHASH function is quite
capable of coming up with its own Tie::IxHash object if it needs one:

Quote:
>> sub TIEHASH {
> ...
>    if (-f $self->[1]) {
>      do $self->[1];
>>     $self->[0] = $PersistentHash;  # XXX no problem any more!
>    } else {
>      $self->[0] = Tie::IxHash->TIEHASH()  # LOD!
>    }
> ...
>> }
>That is a lot to absorb, and I think it will solve your problem, so
>you might want to put this message aside now and come back to the rest
>of it later if you are still interested.

I wrote thart, and then the second method turned out to be easier, but
I did not think to go back and change it.  That is why we have
editors.  (I mean the people here, and not the programs.)

Quote:
>    package My::IxHash;
>    use Tie::IxHash;

>    sub TIEHASH {
>      my $package = shift;
>      my $old_object = shift;
>      # It's already constructed, so just return it!
>      $old_object;          # LOD!
>    }

There is a subtlety here, which is why the line is marked with `LOD'.
(`LOD' is the Hungarian word for `subtle feature'.)  The function
appears to be a constructor for the My::IxHash package, but it does
*not* construct My::IxHash objects.  Instead, it constructs
Tie::IxHash objects.  That is probably a violation of class boundaries
or something, but I don't care; it saves having to do inheritance in a
null subclass, which is a waste of time.

If it bothers you, a solution is to add your own constructor to the
existing Tie::IxHash package.  You can do this by putting the
following into your main program (or anywhere else):

        sub Tie::IxHash::newconstructor {
           # ... as before ..
        }

This is probably an even grosser violation of class boundaries, but if
you won't tell I won't either.



Wed, 14 Nov 2001 03:00:00 GMT  
 Tough one: how to chain ties?

Quote:

> >  tie %a, Tie::IxHash;
> >  tie %b, Tie::PersistentHash, $filename, \%a;

> Oh, this is an interesting approach.

Thanks. I was fascinated by the idea myself. Maybe this
can set a new standard for reusability in Tie:: packages...

Quote:
> The first thing I notice here is that you have the arguments to
> TIEHASH wrong.  The first argument will be the class name, not the
> filename.  So you needed to have
> ...
> I guess you got this right in your implementation and forgot to
> include it only in the news article?  Because actually I think it
> needs to look more like this:

Yep. forgot the `bless $self => shift' at the top. BTW, is there
a reason to always put the bless statement as the last statement in
object constructors? I can understand the need when subclassing, but
in simple cases?

Quote:
> There are a couple of ways around it.  One way is that the object
> doesn't actually have to be tied to anything to function as an
> interface to a tied hash class.  Forget about %a completely, and just
> use the IxHash object directly.  

Very good suggestion. Look below for the solution I found...

Quote:
> Another, totally different strategy is to make a class just like
> Tie::IxHash, except that instead of creating a new object from
> scratch, it will let you specify and old object as an initializer.
> Then you can read the old object out of the file with Data::Dumper and
> hand it off to the initializer for the new class.

The problem is, I don't want to mention Tie::IxHash explicitely
to keep it open for whatever other Tie:: package might be
created in the future (to be true to the perl spirit).

I took another stab at it and came up with the following,
which (I think) does the same as the suggested extension

Quote:
> >  tie %a, $xref;

I dump out the data with

  open DB, ">$self->[1]";
  if (tied %{$self->[0]}) {
    print DB Data::Dumper->Dump([tied(%{$self->[0]})], [qw(PersistentHash)]);
  } else {
    print DB Data::Dumper->Dump([$self->[0]], [qw(PersistentHash)]);
  }
  close DB;

This also stores the type of the tied hash. Now I can read it back
in and get a hash (in $PersistentHash) blessed to whatever
package implements the chained tied hash. Then I recreate the
tied package with

  if (ref($PersistentHash) eq 'HASH') {
    $self->[0] = $PersistentHash; # for unchained case
  } else {
    my %h;
    my $hr = tie %h, ref($PersistentHash); # recreate the tie
    $self->[0] = \%h;
    # now copy the read object over the empty tie...
    if ($hr->isa('ARRAY')) {

    } elsif ($hr->isa('HASH')) {
      %$hr = %$PersistentHash;
    } elsif ($hr->isa('SCALAR')) {
      $$hr = $$PersistentHash;
    } elsif ($hr->isa('GLOB')) {
      *$hr = *$PersistentHash;
    } else {
      die "Cannot handle object type $hr";
    }
  }

and Viola! (tm)

BTW, is there an easier way to do the copy? I thought that a glob
should take care of it, but experiments show not...

Now just some testcases and it should be ready for a first release
(but I'll put in Storable as an alternative for Data::Dumper
for speed)...

Roland
--

ALCATEL Austria, Dept. RTPM       FAX:   +43-1-27722-3955
Scheydgasse 41, A-1210 WIEN, Austria (no Kangaroos here!)



Fri, 16 Nov 2001 03:00:00 GMT  
 Tough one: how to chain ties?

Quote:



>>        my $a = (tie {} => Tie::IxHash);
>>        tie %b => Tie::PersistentHash, $filename, $a;

>This won't work; you'd have to do something like

>    my $a = do { my %a; tie %a => Tie::IxHash };
>    tie %b => Tie::PersistentHash, $filename, $a;

Only because you've got a hashref instead of a hash; you don't need a
named variable or a do{}.  This ought to work

        my $a = tie %{+{}} => Tie::IxHash;
        tie %b => Tie::PersistentHash, $filename, $a;

(and maybe you don't need that + ).    And of course you can flatten it to

     tie %b => Tie::PersistentHash, $filename, tie %{+{}} => Tie::IxHash;

Mike Guy



Sat, 17 Nov 2001 03:00:00 GMT  
 Tough one: how to chain ties?

Quote:
> It would be nice if

>  tie %a, $xref;

> would do the right thing, i.e. tie the variable to the module
> $xref is blessed into AND initialize it with the contents of $xref.

> Perl Gods, maybe in v5.006?

That feature turns out to be unnecessary.  Here's how you do it:

        sub Rebind::TIEHASH { $_[1] }

        tie %a, Rebind, $xref;

That does just what you asked for.  The hash %a is now tied to the
pre-existing object $xref.  Stores and fetches on %a will turn into
method calls in $xref's package.

Here's another possible application of this: You have some tied hash
class X, and you want to tie three hashes into it, but for some reason
you want all three hashes to share the same underlying object.  Here's
how you could do that:

        sub Rebind::TIEHASH { $_[1] }

        tie %a => X, ...... ;
        tie %b => Rebind, (tied %a);
        tie %c => Rebind, (tied %a);

Now fetches and stores on any of %a, %b, and %c are all dispatched
through the same object from class X.  I can't think of any uses for
this offhand, but it does seem useful, and now that I've said how to
do it maybe people will find uses for it.



Thu, 29 Nov 2001 03:00:00 GMT  
 Tough one: how to chain ties?

Quote:

> > It would be nice if

> >  tie %a, $xref;

> > would do the right thing, i.e. tie the variable to the module
> > $xref is blessed into AND initialize it with the contents of $xref.

> > Perl Gods, maybe in v5.006?

> That feature turns out to be unnecessary.  Here's how you do it:

>         sub Rebind::TIEHASH { $_[1] }

>         tie %a, Rebind, $xref;

Wow!

Now this is what I call a real hack:

It's short, easy to understand, yet not obvious and it makes
my previous solution (copying the object data around) look
clumsy, inelegant and plain dumb...

Long live the Power of Perl!

Roland
PS: That really makes you wonder how much more elegant solutions
exist in perl that nobody has even found the *problem* for... ;-)
--


lusternohack /,$0#eg; print; '        -- Roland Giersig, JAPH.



Fri, 30 Nov 2001 03:00:00 GMT  
 
 [ 8 post ] 

 Relevant Pages 

1. Tough one: Running a script as another user

2. tough one.

3. Tough one: Running a script as another user

4. Here's a tough one (relating to 500 error)

5. tying a widget in one canvas to the scrolling of another canvas

6. Perl5 method chaining

7. what command do i use to chain to another perl program

8. chained maps and greps

9. help w chained "maps"

10. || operation with command chaining.

11. Markov chain generator available?

12. Chaining =~'s

 

 
Powered by phpBB® Forum Software