half object/half dictionary 
Author Message
 half object/half dictionary

I wrote a metaclass(?) for manipulating a class with __slots__ as a
dictionary.  It is to avoid sections of code like this:

abc.customer = xyz.customer
abc.work_order = xyz.work_order
abc.purchase_order = xyz.purchase_order
(20 more lines)

Instead I would do this:


Also I can grab all the attribute names, and create a table on the fly
with those attribute names as column names.

I like "abc.customer" better than "abc['customer']" for the obvious
reason, and also because it lets me use PythonWin's
auto-attribute-completion when I type the period.

I might use it like this:

class stockroom_part_class(HalfObjectHalfDict):
    __slots__ = ( 'stock_class_code', 'part_number', 'inventory_type',
                  'mfg_or_purch', 'part_accounting_value',
                  'part_current_revision', 'part_description',
                  'part_status', 'product_family', 'product_model',
                  'qty_onhand', 'qty_used_last_year',
                  'qty_used_this_year', 'allocated_manufacturing',
                  'first_component_part', 'made_on_assembly',
                  'first_routing_sequence', 'last_routing_sequence',
                  'has_routing', 'stocking_unit_of_measure',
                  'bin_item', 'safety_stock')

As you can guess, I want to automate the code for all these attributes
as much as possible.

The code for this "metaclass" is at the end of this post.

Anyway, is this really a "metaclass"?  I guess I should be using
"__metaclass__", but I really don't understand the difference between
setting "__metaclass__" and regular subclassing.  I don't understand
the difference between "__init__" and "__new__".

Does it make sense for a class to have more than one "__metaclass__"

Metaclasses typically subclass 'type', right?

I haven't found any documentation on python metaclasses that I
understand.  What is everyone's favorite reference on Python


(code follows)

class HalfObjectHalfDict(object):
    """Subclass to make object you can operate on like a dictionary.
    Assumes you are setting __slots__

    __slots__ = ()
    def __init__(self, *args0, **args1):
        self.Update(*args0, **args1)
    def __len__(self):
        return len(self.SlotsDict())
    def __contains__(self, item):
        return item in self.SlotsDict()
    def __iter__(self):
        return iter(self.SlotsDict())
    def __getitem__(self, key):
        if self.IsInSlots(key):
                value = self.__getattribute__(key)
            except AttributeError:
                raise KeyError(key)
                return value
            raise KeyError(key)
    def __setitem__(self, key, value):
        if not self.IsInSlots(key):
            raise AttributeError
        self.__setattr__(key, value)
    def Slots(self):
        return self.__class__.__slots__
    def IsInSlots(self, name):
        return name in self.__class__.__slots__
    def SetOnlySlot(self, name, value):
        if not self.IsInSlots(name): return
        self.__setattr__(name, value)
    def UpdateFromDict(self, d):
        for key,value in d.items():
            self.SetOnlySlot(key, value)
    def Update(self, *args0, **args1):
        for a in args0: self.UpdateFromDict(a)
    def SlotsDict(self):
        d = {}
        for name in self.Slots():
                value = self.__getattribute__(name)
            except AttributeError:
                d[name] = value
        return d
    def has_key(self, k):
        return self.SlotsDict().has_key(k)
    def keys(self):
        return self.SlotsDict().keys()
    def values(self):
        return self.SlotsDict().values()
    def items(self):
        return self.SlotsDict().items()
    def copy(self):
        return copy.copy(self)
    def update(self, d):
    def get(self, key, default=None):
        return self.SlotsDict().get(key, default)

Sat, 28 May 2005 03:24:47 GMT  
 half object/half dictionary


> The code for this "metaclass" is at the end of this post.

> Anyway, is this really a "metaclass"?  I guess I should be using
> "__metaclass__", but I really don't understand the difference between
> setting "__metaclass__" and regular subclassing.  I don't understand
> the difference between "__init__" and "__new__".

It isn't a metaclass.

Simply put, a metaclass (or metatype, they are interchangable) is a
type who's instances are types.

(And, to be specific, a type is any Python object that is capable of
having instances, whereas "type" is a certain Python object that all
Python built-in types are instances of.  "type" is a metatype.)

To understand, on the surface, what __metaclass__ does, you have to
think about what the class statement does, ignoring old-style classes
for the moment.  What the class statements really does is ask the
"type" object to create an instance of itself.  It does this by
calling the object with three arguments: the class's name, a tuple of
it's bases, and a dict of the class attributes.  Essentially, the
following class definition:

    class spam(object):
        a = 1
        def b: pass

is equivalent to the following assignment:

    spam = type("spam", (object,), {'a': 1, 'b': <function b at 0x80d3a14>})

What "type" does, when called this way, is to create an instance of
itself (initializing the name, bases, and dict using the arguments).
Instances of "type" are, of course, types, and that's how the class
statement creates types.

When you use __metaclass__ = some_other_object, you are telling Python
that you want the some_other_object (instead of "type") to handle the
creation of this object.  Therefore, the following class definition:

    class spam(object):
        __metaclass__ = some_other_object
        a = 1

is equivalent to this assignment:

    spam = some_other_object("spam", (object,),
                             {'a': 1, '__metaclass__': some_other_object)

If some_other_object is a metaclass (it doesn't have to be), it
typically returns an instance of itself, as "type" does.  However, it
is free to create an object with radically different behavior than an
instance of "type".  And instances of that object (which is a type,
since it is an instance of a metaclass) will have radically different
behavior also.

Confuesed yet?

However, this is only the beginning.  Understanding what use you can
put that power to is quite hard; I am only beginning to get a feel for
what I can do with metaclasses.

> Does it make sense for a class to have more than one "__metaclass__"
> set?

> Metaclasses typically subclass 'type', right?

Yes, because instances of metaclasses are types.  In C, you could
write a class that does not subclass type but is a metaclass; but in
Python, a metaclass has to be a subclass of another metaclass.

The only metaclass built into Python (for now), is, of course, type,
so that's what Python metaclasses typically subclass.

> I haven't found any documentation on Python metaclasses that I
> understand.  What is everyone's favorite reference on Python
> metaclasses?

Don't feel too bad.  I've had a hard time getting my head around this
metaclasses stuff, and I usually tackle new concepts easily.  This is
hard stuff to get a feel for.

I basically learned metaclasses from the PEPs, so I can't help you
much.  There is a page on the Python site that shows how to use
old-style classes as metaclasses.


It wasn't much help in learning the details of how to use metaclasses
with new-style clasess, but it was helpful in getting me a feel for
what metaclasses could do.

If you're a visual learner, I suggest drawing a graph showing type,
instance, and subclass relationships.  Use boxes to represent
_objects_, and arrows to represent type-of, instance-of, and

> (code follows)

> class HalfObjectHalfDict(object):
>    """Subclass to make object you can operate on like a dictionary.
>    Assumes you are setting __slots__

>    """
>    #
>    __slots__ = ()
>    #
>    def __init__(self, *args0, **args1):
>        self.Update(*args0, **args1)
>    #
>    def __len__(self):
>        return len(self.SlotsDict())
>    #
>    def __contains__(self, item):
>        return item in self.SlotsDict()
>    #
>    def __iter__(self):
>        return iter(self.SlotsDict())
>    #
>    def __getitem__(self, key):
>        if self.IsInSlots(key):
>            try:
>                value = self.__getattribute__(key)
>            except AttributeError:
>                raise KeyError(key)
>            else:
>                return value
>        else:
>            raise KeyError(key)
>    #
>    def __setitem__(self, key, value):
>        if not self.IsInSlots(key):
>            raise AttributeError
>        self.__setattr__(key, value)
>    #
>    def Slots(self):
>        return self.__class__.__slots__
>    #
>    def IsInSlots(self, name):
>        return name in self.__class__.__slots__
>    #
>    def SetOnlySlot(self, name, value):
>        if not self.IsInSlots(name): return
>        self.__setattr__(name, value)
>    #
>    def UpdateFromDict(self, d):
>        for key,value in d.items():
>            self.SetOnlySlot(key, value)
>    #
>    def Update(self, *args0, **args1):
>        for a in args0: self.UpdateFromDict(a)
>        self.UpdateFromDict(args1)
>    #
>    def SlotsDict(self):
>        d = {}
>        for name in self.Slots():
>            try:
>                value = self.__getattribute__(name)
>            except AttributeError:
>                pass
>            else:
>                d[name] = value
>        return d
>    #
>    def has_key(self, k):
>        return self.SlotsDict().has_key(k)
>    #
>    def keys(self):
>        return self.SlotsDict().keys()
>    #
>    def values(self):
>        return self.SlotsDict().values()
>    #
>    def items(self):
>        return self.SlotsDict().items()
>    #
>    def copy(self):
>        return copy.copy(self)
>    #
>    def update(self, d):
>        self.UpdateFromDict(d)
>    #
>    def get(self, key, default=None):
>        return self.SlotsDict().get(key, default)

I guess it'll work, but it seems highly inefficient to me.  You are
creating a new dict every call to SlotsDict, which seems to happen
every call.

But you don't really have to create the dict; you can use getattr and
setattr and __slots__.  I'll give you an example:

class HalfObjectHalfDict(object):
    __slots__ = ()

    def keys(self):
        return self.__slots__

    def values(self):
        return [ getattr(self,x) for x in self.__slots__ ]

    def items(self):
        return [ (x,getattr(self,x)) for x in self.__slots__ ]

    def __getitem__(self,attr):
        return getattr(self,attr)

    def __setitem__(self,attr,value):
        return setitme(self,attr,value)

    def update(self,other):
        for attr in other.__slots__:



Sat, 28 May 2005 05:53:35 GMT  
 half object/half dictionary
Thanks so much!

It is becoming clearer (still hazy, but better)

My mind still boggles at why we would need 'cls' in the following

class meta1(type):
        def __new__(cls, classname, bases, classdict):
                return type.__new__(cls, classname, bases, classdict)

Here is an example of a metaclass where 'cls' is used and not just
passed along to type.__new__ :


but I can't say I really understand this.

Does type.__new__(cls, classname, bases, classdict) do anything with


Sat, 28 May 2005 07:53:07 GMT  
 half object/half dictionary


> I wrote a metaclass(?) for manipulating a class with __slots__ as a
> dictionary.  It is to avoid sections of code like this:

In case you are interested, here's my take on the same problem. Feel free to
use any of the code for your purposes. My Struct class mimics as much as
possible a dict but it allows named attribute access.



# File Struct.py

"""Mimic C structs with lots of extra functionality."""


#  Distributed under the terms of the GNU Lesser General Public License (LGPL)
#    This code is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    Lesser General Public License for more details.
#  The full text of the LGPL is available at:
#                  http://www.gnu.org/copyleft/lesser.html

__version__ = '0.1.0'
__license__ = 'LGPL'
__date__   = 'Tue Dec 11 00:27:58 MST 2001'

__all__ = ['Struct']

import types
from genutils import list2dict2

class Struct:
    """Class to mimic C structs but also provide convenient dictionary-like

    Instances can be initialized with a dictionary, a list of key=value pairs
    or both. If both are present, the dictionary must come first.

    Because Python classes provide direct assignment to their members, it's
    easy to overwrite normal methods (S.copy = 1 would destroy access to
    S.copy()). For this reason, all builtin method names are protected and
    can't be assigned to. An attempt to do s.copy=1 or s['copy']=1 will raise
    a KeyError exception. If you really want to, you can bypass this
    protection by directly assigning to __dict__: s.__dict__['copy']=1 will
    still work. Doing this will break functionality, though. As in most of
    Python, namespace protection is weakly enforced, so feel free to shoot
    yourself if you really want to.

    Note that this class uses more memory and is *much* slower than a regular
    dictionary, so be careful in situations where memory or performance are
    critical. But for day to day use it should behave fine. It is particularly
    convenient for storing configuration data in programs.

    +,+=,- and -= are implemented. +/+= do merges (non-destructive updates),
    -/-= remove keys from the original. See the method descripitions.

    This class allows a quick access syntax: both s.key and s['key'] are
    valid.  This syntax has a limitation: each 'key' has to be explicitly
    accessed by its original name. The normal s.key syntax doesn't provide
    access to the keys via variables whose values evaluate to the desired
    keys. An example should clarify this:

    Define a dictionary and initialize both with dict and k=v pairs:
    >>> d={'a':1,'b':2}
    >>> s=Struct(d,hi=10,ho=20)
    The return of __repr__ can be used to create a new instance:
    >>> s
    Struct({'ho': 20, 'b': 2, 'hi': 10, 'a': 1})
    __str__ (called by print) shows it's not quite a regular dictionary:
    >>> print s
    Struct {a: 1, b: 2, hi: 10, ho: 20}
    Access by explicitly named key with dot notation:
    >>> s.a
    Or like a dictionary:
    >>> s['a']
    If you want a variable to hold the key value, only dictionary access
    >>> key='hi'
    >>> s.key
    Traceback (most recent call last):
      File "<stdin>", line 1, in ?
    AttributeError: Struct instance has no attribute 'key'
    >>> s[key]

    Another limitation of the s.key syntax (and Struct(key=val)
    initialization): keys can't be numbers. But numeric keys can be used and
    accessed using the dictionary syntax. Again, an example:

    This doesn't work:
    >>> s=Struct(4='hi')
    SyntaxError: keyword can't be an expression
    But this does:
    >>> s=Struct()
    >>> s[4]='hi'
    >>> s
    Struct({4: 'hi'})
    >>> s[4]

    # Attributes to which __setitem__ and __setattr__ will block access.
    # Note: much of this will be moot in Python 2.2 and will be done in a much
    # cleaner way.
    __protected = ('copy dict dictcopy get has_attr has_key items keys '
                   'merge popitem setdefault update values '
                   '__make_dict __dict_invert ').split()

    def __init__(self,dict=None,**kw):
        """Initialize with a dictionary, another Struct, or by giving
        explicitly the list of attributes.

        Both can be used, but the dictionary must come first:
        Struct(dict), Struct(k1=v1,k2=v2) or Struct(dict,k1=v1,k2=v2).
        if dict is None:
            dict = {}
        if isinstance(dict,Struct):
            dict = dict.dict()
        elif dict and  type(dict) is not types.DictType:
            raise TypeError,\
                  'Initialize with a dictionary or key=val pairs.'
        # do the updating by hand to guarantee that we go through the
        # safety-checked __setitem__
        for k,v in dict.items():
            self[k] = v

    def __setitem__(self,key,value):
        """Used when struct[key] = val calls are made."""
        if key in Struct.__protected:
            raise KeyError,'Key '+`key`+' is a protected key of class Struct.'
        self.__dict__[key] = value

    def __setattr__(self, key, value):
        """Used when struct.key = val calls are made."""

    def __str__(self):
        """Gets called by print."""

        return 'Struct('+str(self.__dict__)+')'

    def __repr__(self):
        """Gets called by repr.

        A Struct can be recreated with S_new=eval(repr(S_old))."""
        return 'Struct('+str(self.__dict__)+')'

    def __getitem__(self,key):
        """Allows struct[key] access."""
        return self.__dict__[key]

    def __contains__(self,key):
        """Allows use of the 'in' operator."""
        return self.__dict__.has_key(key)

    def __iadd__(self,other):
        """S += S2 is a shorthand for S.merge(S2)."""
        return self

    def __add__(self,other):
        """S + S2 -> New Struct made form S and S.merge(S2)"""
        Sout = self.copy()
        return Sout

    def __sub__(self,other):
        """Return S1-S2, where all keys in S2 have been deleted (if present)
        from S1."""
        Sout = self.copy()
        Sout -= other
        return Sout

    def __isub__(self,other):
        """Do in place S = S - S2, meaning all keys in S2 have been deleted
        (if present) from S1."""

        for k in other.keys():
            if self.has_key(k):
                del self.__dict__[k]

    def __make_dict(self,__loc_data__,**kw):
        "Helper function for update and merge. Return a dict from data."

        if __loc_data__ == None:
            dict = {}
        elif type(__loc_data__) is types.DictType:
            dict = __loc_data__
        elif isinstance(__loc_data__,Struct):
            dict = __loc_data__.__dict__
            raise TypeError, 'Update with a dict, a Struct or key=val pairs.'
        if kw:
        return dict

    def __dict_invert(self,dict):
        """Helper function for merge. Takes a dictionary whose values are
        lists and returns a dict. with the elements of each list as keys and
        the original keys as values."""

        outdict = {}
        for k,lst in dict.items():
            if type(lst) is types.StringType:
                lst = lst.split()
            for entry in lst:
                outdict[entry] = k
        return outdict

    def clear(self):
        """Clear all attributes."""

    def copy(self):
        """Return a (shallow) copy of a Struct."""
        return Struct(self.__dict__.copy())

    def dict(self):
        """Return the Struct's dictionary."""
        return self.__dict__

    def dictcopy(self):
        """Return a (shallow) copy of the Struct's dictionary."""
        return self.__dict__.copy()

    def popitem(self):
        """S.popitem() -> (k, v), remove and return some (key, value) pair as
        a 2-tuple; but raise KeyError if S is empty."""
        return self.__dict__.popitem()

    def update(self,__loc_data__=None,**kw):
        """Update (merge) with data from another Struct or from a dictionary.
        Optionally, one or more key=value pairs can be given at the end for
        direct update."""

        # The funny name __loc_data__ is to prevent a common variable name
        # could be a fieled of a Struct to collide with this parameter. The
        # would arise if the function is called with a keyword with this same
        # that a user means to add as a Struct field.
        newdict = Struct.__make_dict(self,__loc_data__,**kw)
        for k,v in newdict.items():
            self[k] = v

    def merge(self,__loc_data__=None,__conflict_solve=None,**kw):
        """S.merge(data,conflict,k=v1,k=v2,...) -> merge data and k=v into S.

        This is similar to update(), but much more flexible.  First, a dict is
        made from data+key=value pairs. When merging this dict with the Struct
        S, the optional dictionary 'conflict' is used to decide what to do.

        If conflict is not given, the default behavior is to preserve any keys
        with their current value (the opposite of the update method's

        conflict is a dictionary of binary functions which will be used to
        solve key conflicts. It must have the following structure:

          conflict == { fn1 : [Skey1,Skey2,...], fn2 :

read more »

Sat, 28 May 2005 08:21:19 GMT  
 half object/half dictionary
Thanks for the code! The "merge" method looks like a very good idea.


Sat, 28 May 2005 09:23:26 GMT  
 [ 5 post ] 

 Relevant Pages 

1. lsort Half Numerical Half Text

2. Enhancement suggestion (half-baked)

3. Looking how to draw a half circle

4. Database half-baked?

5. Loop Question - Only getting half the records

6. C4b ABC - Report shifts half a page to the right

7. Closing window half way through OK button using ABC/Update Templates

8. ANN:Get gReg at HALF PRICE!

9. Sather 0.5: half way to Sather 1.0 -- sort of

10. Half length printing

11. RS-485 Half Duplex and Serial vi's

12. Moving printer half line?


Powered by phpBB® Forum Software