Playing with Dict - Can anyone see a way around this? 
Author Message
 Playing with Dict - Can anyone see a way around this?

Somebody mentioned in this group a couple of months ago, the posibility
of making [].keys, [].items and [].values directly scriptable, so you
could iterate over them either by calling them, or by treating them as
lists. I've taken their suggestion as far as I can go as an exercise in
curiosity, but have run into a little bit of a problem. I took their
suggestion of making keys, values and items, classes rather than
functions.

class MyDict:
        def __init__(self):
                self.dict = {}
                self.keys = self.__keys(self.dict)
                self.items = self.__items(self.dict)
                self.values = self.__values(self.dict)
        def __getitem__(self,item):
                return self.dict[item]
        def __setitem__(self,item,value):
                self.dict[item] = value
        def __getattr__(self,key):
                if key in ['keys','items','values']:
                        return self.__dict__[key]
                else:
                        return self.dict[key]
        def __repr__(self):
                return str(self.dict)
        def __delitem__(self,item):
                del self.dict[item]
        class __keys:
                def __init__(self,dict):
                        self.dict = dict
                def __repr__(self):
                        return str(self.dict.keys())
                def __call__(self):
                        return self.dict.keys()
                def __getitem__(self,item):
                        return self.dict.keys()[item]
        class __items:
                def __init__(self,dict):
                        self.dict = dict
                def __repr__(self):
                        return str(self.dict.items())
                def __call__(self):
                        return self.dict.items()
                def __getitem__(self,item):
                        return self.dict.items()[item]
        class __values:
                def __init__(self,dict):
                        self.dict = dict
                def __repr__(self):
                        return str(self.dict.values())
                def __call__(self):
                        return self.dict.values()
                def __getitem__(self,item):
                        return self.dict.values()[item]

Now this mostly works

Quote:
>>> a = MyDict()
>>> for i in range(10):

        a[i] = i

Quote:
>>> a

{9: 9, 8: 8, 7: 7, 6: 6, 5: 5, 4: 4, 3: 3, 2: 2, 1: 1, 0: 0}
Quote:
>>> a.keys

[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
Quote:
>>> a.values

[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]
Quote:
>>> a.items

[(9, 9), (8, 8), (7, 7), (6, 6), (5, 5), (4, 4), (3, 3), (2, 2), (1, 1),
(0, 0)]

And it's possible to iterate over a.items in the following ways:

Quote:
>>> for i in a.keys:

        print i,

9 8 7 6 5 4 3 2 1 0

Quote:
>>> for i in a.keys():

        print i,

9 8 7 6 5 4 3 2 1 0

Quote:
>>> for i in a.items():

        print i,

(9, 9) (8, 8) (7, 7) (6, 6) (5, 5) (4, 4) (3, 3) (2, 2) (1, 1) (0, 0)

However, I've hit a rather interesting hiccup, due to i.keys returning
the class __keys, rather than a list of keys. Umm, it's explained better
in the example.

Now everything looks okay at first glance, due to some hacking on
__keys.__repr__

Quote:
>>> b = a.keys
>>> b

[9, 8, 7, 6, 5, 4, 3, 2, 1, 0]

But watch what happens when I iterate over a.keys and change the
dictionary in place.

Quote:
>>> for i in a.keys:

        print i
        del a[i]

9
7
5
3
1

Quote:
>>> a

{8: 8, 6: 6, 4: 4, 2: 2, 0: 0}

Whoops. Not what I wanted. And this invalidates my aim of having a.keys
and a.keys() acting the same way.

Quote:
>>> for i in a.keys():

        print i
        del a[i]

9
8
7
6
5
4
3
2
1
0

Anycase, can anyone see a way around this? This is mostly for curiosity,
but I'm damned curious at the moment.

Joal Heagney/AncientHart



Thu, 06 Nov 2003 00:08:57 GMT  
 Playing with Dict - Can anyone see a way around this?

    ...

Quote:
> def __getattr__(self,key):
>     if key in ['keys','items','values']:
>         return self.__dict__[key]

Note that this part is unnecessary: __getattr__ is never called
for an attribute name that is found in the obj's __dict__ (it's
different for __setattr__).

Quote:
>         class __keys:
>                 def __init__(self,dict):
>                         self.dict = dict
>                 def __repr__(self):
>                         return str(self.dict.keys())
>                 def __call__(self):
>                         return self.dict.keys()
>                 def __getitem__(self,item):
>                         return self.dict.keys()[item]

Simpler and maybe faster:
            class __keys:
                def __init__(self, dict):
                    self.keys = dict.keys()
                def __repr__(self):
                    return repr(self.keys)
                def __call__(self):
                    return self.keys
                def __getitem__(self, item):
                    return self.keys[item]

Calling self.dict.keys() repeatedly for each use should be slower
than calling it once and caching the result.  Besides:

Quote:
> Whoops. Not what I wanted. And this invalidates my aim of having a.keys
> and a.keys() acting the same way.
> >>> for i in a.keys():
> print i
> del a[i]

now 'for i in a.keys' will also let you alter a during the loop.

An optional little enhancement is to add a second line to the
__keys' __init__:
                    self.keys.sort()
The sort shouldn't slow things down by much, and often it's
nicer to see the keys in sorted order:-).

This however would not keep the property of corresponding
order between keys, items and values (when called without
intervening alterations to the dictionary).  But that is not
used very often anyway.

Alex



Wed, 05 Nov 2003 15:43:52 GMT  
 Playing with Dict - Can anyone see a way around this?
Quote:

> Somebody mentioned in this group a couple of months ago, the
> posibility
> of making [].keys, [].items and [].values directly scriptable, so you
> could iterate over them either by calling them, or by treating them as
> lists. I've taken their suggestion as far as I can go as an exercise
> in
> curiosity, but have run into a little bit of a problem. I took their
> suggestion of making keys, values and items, classes rather than
> functions.

From what I understand you should check out the looping PEP on the
python website. Looping through dictionaries are scheduled for the next
release.

Charlie



Wed, 05 Nov 2003 22:05:20 GMT  
 Playing with Dict - Can anyone see a way around this?
Thanks Alex. I had to play around with the MyDict class to force update
of the keys, but this works as I wanted it to. I decided to leave out
the sort - mainly because of the desync between keys,values. (Items
would still be synched with keys, am I right?) Now I really should think
about wrapping these in UserDict and it's pals.

class MyDict:
        def __init__(self):
                self.dict = {}
                self.__update()
        def __getitem__(self,item):
                return self.dict[item]
        def __setitem__(self,item,value):
                self.dict[item] = value
                self.__update()
        def __update(self):
                self.keys = self.__keys(self.dict)
                self.items = self.__items(self.dict)
                self.values = self.__values(self.dict)
        def __getattr__(self,key):
                return self.dict[key]
        def __repr__(self):
                return str(self.dict)
        def __delitem__(self,item):
                del self.dict[item]
                self.__update()
        class __keys:
                def __init__(self,dict):
                        self.keys = dict.keys()
                def __repr__(self):
                        return repr(self.keys)
                def __call__(self):
                        return self.keys
                def __getitem__(self,item):
                        return self.keys[item]
        class __items:
                def __init__(self,dict):
                        self.items = dict.items()
                def __repr__(self):
                        return repr(self.items)
                def __call__(self):
                        return self.items
                def __getitem__(self,item):
                        return self.items[item]
        class __values:
                def __init__(self,dict):
                        self.keys = dict.values()
                def __repr__(self):
                        return repr(self.values)
                def __call__(self):
                        return self.values
                def __getitem__(self,item):
                        return self.values[item]

Charlie, I've had a look at the iterator PEP, and I like the idea of
optimised iterators for dictionaries, but I also like the original post
(months ago) that inspired this code, in that you don't have to remember
any new names for iteration over items/values etc. *shrugs* I'm not
particularly bothered how it's done, I was curious on how to do this,
and hey, I discovered that python doesn't use __getattr__ if the
attribute is in __dict__.

Joal Heagney/AncientHart



Thu, 06 Nov 2003 17:29:25 GMT  
 Playing with Dict - Can anyone see a way around this?

Quote:
> Thanks Alex. I had to play around with the MyDict class to force update
> of the keys, but this works as I wanted it to. I decided to leave out
> the sort - mainly because of the desync between keys,values. (Items
    ...
> def __update(self):
>     self.keys = self.__keys(self.dict)
>     self.items = self.__items(self.dict)
>     self.values = self.__values(self.dict)

This does allow dictionary alteration during a loop on (e.g.) keys
(it's gonna be pretty slow since __update will be called over and
over again, but that's another issue) because a shiny new list is
prepared for (e.g.) self.keys each time, rather than altering the
old one... and the loop will be looping on the old one.  So, fine
(apart from possible performance issues).

Sorting isn't hard if you do want it, e.g.:

    self.items = self.__items(self.dict)
    self.items.sort()
    self.keys, self.values = map(list,zip(*self.items))

Quote:
> and hey, I discovered that python doesn't use __getattr__ if the
> attribute is in __dict__.

Excellent, because this is absolutely key.  In particular, it is what
allows a potentially important optimization regarding the above
mentioned performance issue.  Consider some client-code...:

    for k in plik.keys:
        if buup(k): del plik[k]
    for k in plik.keys:
        print 'Survived key:',k

In the current approach, plik.__update will be called at each and
every del -- that may be a lot of times and a lot of work... all for
nothing except the last time.

This is a reasonably frequent pattern of accesses to data: when a
change comes, many changes may come 'bunched up', then no
more changes for a while (just read-access) until next bunch.

Which is why motivates lazy, aka just-in-time, strategies of
updating... consider:

def __recompute(self, attname):
    self.items = self.__items(self.dict)
    self.items.sort()
    self.keys, self.values = map(list,zip(*self.items))
    return getattr(self, attname)

def __update(self):
    del self.items, self.keys, self.values

def __getattr__(self, attname):
    if attname in ('items','keys','values'):
        return self.__recompute(attname)
    # continue with the rest of __getattr__ as now

When the underlying dict changes, __update is called, just
like now, but what it does is "mark invalid" the attributes by
the simple and effective strategy of removing them altogether.

So NEXT time an attribute is accessed, it won't be in __dict__
and so __getattr__ will be called -- and it will delegate to
__recompute for just-in-time recomputation...

Alex



Thu, 06 Nov 2003 15:18:13 GMT  
 Playing with Dict - Can anyone see a way around this?

Quote:
> So NEXT time an attribute is accessed, it won't be in __dict__
> and so __getattr__ will be called -- and it will delegate to
> __recompute for just-in-time recomputation...

> Alex

Now THAT I like. I was concerned about the efficiency of updating lists
every time I added or subtracted from the list. I haven't had much
experience with map, and zip, so while I'm using the code, I'm also
reading up a little on these commands.

Joal Heagney/AncientHart



Sat, 08 Nov 2003 05:51:54 GMT  
 
 [ 6 post ] 

 Relevant Pages 

1. I have seen the error of my ways!

2. VisualWave-playing around

3. Playing around with continuations

4. Playing around with midi + python

5. (VW)Anyone seen a RangeDictionary or an OrderedDictionary?

6. VWave Server 2.0 beta...Has anyone seen it???

7. CFW 2.003 Has anyone seen this...

8. Anyone saw Gus??

9. Anyone seen Dwiz ???

10. VO2: Has anyone seen my Tool Palette?

11. Weird situation - anyone ever seen it?!

12. Has anyone seen DBU for windows?

 

 
Powered by phpBB® Forum Software