half object/half dictionary
Author |
Message |
Manuel M. Garci #1 / 5
|
 half object/half dictionary
I wrote a metaclass(?) for manipulating a class with __slots__ as a dictionary. It is to avoid sections of code like this: abc.customer = xyz.customer abc.work_order = xyz.work_order abc.purchase_order = xyz.purchase_order (20 more lines) Instead I would do this: abc.update(xyz) Also I can grab all the attribute names, and create a table on the fly with those attribute names as column names. I like "abc.customer" better than "abc['customer']" for the obvious reason, and also because it lets me use PythonWin's auto-attribute-completion when I type the period. I might use it like this: class stockroom_part_class(HalfObjectHalfDict): __slots__ = ( 'stock_class_code', 'part_number', 'inventory_type', 'mfg_or_purch', 'part_accounting_value', 'part_current_revision', 'part_description', 'part_status', 'product_family', 'product_model', 'qty_onhand', 'qty_used_last_year', 'qty_used_this_year', 'allocated_manufacturing', 'first_component_part', 'made_on_assembly', 'first_routing_sequence', 'last_routing_sequence', 'has_routing', 'stocking_unit_of_measure', 'bin_item', 'safety_stock') As you can guess, I want to automate the code for all these attributes as much as possible. The code for this "metaclass" is at the end of this post. Anyway, is this really a "metaclass"? I guess I should be using "__metaclass__", but I really don't understand the difference between setting "__metaclass__" and regular subclassing. I don't understand the difference between "__init__" and "__new__". Does it make sense for a class to have more than one "__metaclass__" set? Metaclasses typically subclass 'type', right? I haven't found any documentation on python metaclasses that I understand. What is everyone's favorite reference on Python metaclasses? Manuel (code follows) class HalfObjectHalfDict(object): """Subclass to make object you can operate on like a dictionary. Assumes you are setting __slots__ """ # __slots__ = () # def __init__(self, *args0, **args1): self.Update(*args0, **args1) # def __len__(self): return len(self.SlotsDict()) # def __contains__(self, item): return item in self.SlotsDict() # def __iter__(self): return iter(self.SlotsDict()) # def __getitem__(self, key): if self.IsInSlots(key): try: value = self.__getattribute__(key) except AttributeError: raise KeyError(key) else: return value else: raise KeyError(key) # def __setitem__(self, key, value): if not self.IsInSlots(key): raise AttributeError self.__setattr__(key, value) # def Slots(self): return self.__class__.__slots__ # def IsInSlots(self, name): return name in self.__class__.__slots__ # def SetOnlySlot(self, name, value): if not self.IsInSlots(name): return self.__setattr__(name, value) # def UpdateFromDict(self, d): for key,value in d.items(): self.SetOnlySlot(key, value) # def Update(self, *args0, **args1): for a in args0: self.UpdateFromDict(a) self.UpdateFromDict(args1) # def SlotsDict(self): d = {} for name in self.Slots(): try: value = self.__getattribute__(name) except AttributeError: pass else: d[name] = value return d # def has_key(self, k): return self.SlotsDict().has_key(k) # def keys(self): return self.SlotsDict().keys() # def values(self): return self.SlotsDict().values() # def items(self): return self.SlotsDict().items() # def copy(self): return copy.copy(self) # def update(self, d): self.UpdateFromDict(d) # def get(self, key, default=None): return self.SlotsDict().get(key, default)
|
Sat, 28 May 2005 03:24:47 GMT |
|
 |
Carl Bank #2 / 5
|
 half object/half dictionary
Quote:
> The code for this "metaclass" is at the end of this post. > Anyway, is this really a "metaclass"? I guess I should be using > "__metaclass__", but I really don't understand the difference between > setting "__metaclass__" and regular subclassing. I don't understand > the difference between "__init__" and "__new__".
It isn't a metaclass. Simply put, a metaclass (or metatype, they are interchangable) is a type who's instances are types. (And, to be specific, a type is any Python object that is capable of having instances, whereas "type" is a certain Python object that all Python built-in types are instances of. "type" is a metatype.) To understand, on the surface, what __metaclass__ does, you have to think about what the class statement does, ignoring old-style classes for the moment. What the class statements really does is ask the "type" object to create an instance of itself. It does this by calling the object with three arguments: the class's name, a tuple of it's bases, and a dict of the class attributes. Essentially, the following class definition: class spam(object): a = 1 def b: pass is equivalent to the following assignment: spam = type("spam", (object,), {'a': 1, 'b': <function b at 0x80d3a14>}) What "type" does, when called this way, is to create an instance of itself (initializing the name, bases, and dict using the arguments). Instances of "type" are, of course, types, and that's how the class statement creates types. When you use __metaclass__ = some_other_object, you are telling Python that you want the some_other_object (instead of "type") to handle the creation of this object. Therefore, the following class definition: class spam(object): __metaclass__ = some_other_object a = 1 is equivalent to this assignment: spam = some_other_object("spam", (object,), {'a': 1, '__metaclass__': some_other_object) If some_other_object is a metaclass (it doesn't have to be), it typically returns an instance of itself, as "type" does. However, it is free to create an object with radically different behavior than an instance of "type". And instances of that object (which is a type, since it is an instance of a metaclass) will have radically different behavior also. Confuesed yet? However, this is only the beginning. Understanding what use you can put that power to is quite hard; I am only beginning to get a feel for what I can do with metaclasses. Quote: > Does it make sense for a class to have more than one "__metaclass__" > set? > Metaclasses typically subclass 'type', right?
Yes, because instances of metaclasses are types. In C, you could write a class that does not subclass type but is a metaclass; but in Python, a metaclass has to be a subclass of another metaclass. The only metaclass built into Python (for now), is, of course, type, so that's what Python metaclasses typically subclass. Quote: > I haven't found any documentation on Python metaclasses that I > understand. What is everyone's favorite reference on Python > metaclasses?
Don't feel too bad. I've had a hard time getting my head around this metaclasses stuff, and I usually tackle new concepts easily. This is hard stuff to get a feel for. I basically learned metaclasses from the PEPs, so I can't help you much. There is a page on the Python site that shows how to use old-style classes as metaclasses. http://www.python.org/doc/essays/metaclasses/ It wasn't much help in learning the details of how to use metaclasses with new-style clasess, but it was helpful in getting me a feel for what metaclasses could do. If you're a visual learner, I suggest drawing a graph showing type, instance, and subclass relationships. Use boxes to represent _objects_, and arrows to represent type-of, instance-of, and subclass-of. Quote: > (code follows) > class HalfObjectHalfDict(object): > """Subclass to make object you can operate on like a dictionary. > Assumes you are setting __slots__ > """ > # > __slots__ = () > # > def __init__(self, *args0, **args1): > self.Update(*args0, **args1) > # > def __len__(self): > return len(self.SlotsDict()) > # > def __contains__(self, item): > return item in self.SlotsDict() > # > def __iter__(self): > return iter(self.SlotsDict()) > # > def __getitem__(self, key): > if self.IsInSlots(key): > try: > value = self.__getattribute__(key) > except AttributeError: > raise KeyError(key) > else: > return value > else: > raise KeyError(key) > # > def __setitem__(self, key, value): > if not self.IsInSlots(key): > raise AttributeError > self.__setattr__(key, value) > # > def Slots(self): > return self.__class__.__slots__ > # > def IsInSlots(self, name): > return name in self.__class__.__slots__ > # > def SetOnlySlot(self, name, value): > if not self.IsInSlots(name): return > self.__setattr__(name, value) > # > def UpdateFromDict(self, d): > for key,value in d.items(): > self.SetOnlySlot(key, value) > # > def Update(self, *args0, **args1): > for a in args0: self.UpdateFromDict(a) > self.UpdateFromDict(args1) > # > def SlotsDict(self): > d = {} > for name in self.Slots(): > try: > value = self.__getattribute__(name) > except AttributeError: > pass > else: > d[name] = value > return d > # > def has_key(self, k): > return self.SlotsDict().has_key(k) > # > def keys(self): > return self.SlotsDict().keys() > # > def values(self): > return self.SlotsDict().values() > # > def items(self): > return self.SlotsDict().items() > # > def copy(self): > return copy.copy(self) > # > def update(self, d): > self.UpdateFromDict(d) > # > def get(self, key, default=None): > return self.SlotsDict().get(key, default)
I guess it'll work, but it seems highly inefficient to me. You are creating a new dict every call to SlotsDict, which seems to happen every call. But you don't really have to create the dict; you can use getattr and setattr and __slots__. I'll give you an example: class HalfObjectHalfDict(object): __slots__ = () def keys(self): return self.__slots__ def values(self): return [ getattr(self,x) for x in self.__slots__ ] def items(self): return [ (x,getattr(self,x)) for x in self.__slots__ ] def __getitem__(self,attr): return getattr(self,attr) def __setitem__(self,attr,value): return setitme(self,attr,value) def update(self,other): for attr in other.__slots__: setattr(self,attr,getattr(other,attr)) ... -- CARL BANKS
|
Sat, 28 May 2005 05:53:35 GMT |
|
 |
Manuel M. Garci #3 / 5
|
 half object/half dictionary
Thanks so much! It is becoming clearer (still hazy, but better) My mind still boggles at why we would need 'cls' in the following statements: class meta1(type): def __new__(cls, classname, bases, classdict): return type.__new__(cls, classname, bases, classdict) Here is an example of a metaclass where 'cls' is used and not just passed along to type.__new__ : http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/160164 but I can't say I really understand this. Does type.__new__(cls, classname, bases, classdict) do anything with 'cls'? Manuel
|
Sat, 28 May 2005 07:53:07 GMT |
|
 |
Fernando Pére #4 / 5
|
 half object/half dictionary
Quote:
> I wrote a metaclass(?) for manipulating a class with __slots__ as a > dictionary. It is to avoid sections of code like this:
In case you are interested, here's my take on the same problem. Feel free to use any of the code for your purposes. My Struct class mimics as much as possible a dict but it allows named attribute access. Cheers, f. ######################################################### # File Struct.py """Mimic C structs with lots of extra functionality.""" #*****************************************************************************
# # Distributed under the terms of the GNU Lesser General Public License (LGPL) # # This code is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # Lesser General Public License for more details. # # The full text of the LGPL is available at: # # http://www.gnu.org/copyleft/lesser.html #*****************************************************************************
__version__ = '0.1.0' __license__ = 'LGPL' __date__ = 'Tue Dec 11 00:27:58 MST 2001' __all__ = ['Struct'] import types from genutils import list2dict2 class Struct: """Class to mimic C structs but also provide convenient dictionary-like functionality. Instances can be initialized with a dictionary, a list of key=value pairs or both. If both are present, the dictionary must come first. Because Python classes provide direct assignment to their members, it's easy to overwrite normal methods (S.copy = 1 would destroy access to S.copy()). For this reason, all builtin method names are protected and can't be assigned to. An attempt to do s.copy=1 or s['copy']=1 will raise a KeyError exception. If you really want to, you can bypass this protection by directly assigning to __dict__: s.__dict__['copy']=1 will still work. Doing this will break functionality, though. As in most of Python, namespace protection is weakly enforced, so feel free to shoot yourself if you really want to. Note that this class uses more memory and is *much* slower than a regular dictionary, so be careful in situations where memory or performance are critical. But for day to day use it should behave fine. It is particularly convenient for storing configuration data in programs. +,+=,- and -= are implemented. +/+= do merges (non-destructive updates), -/-= remove keys from the original. See the method descripitions. This class allows a quick access syntax: both s.key and s['key'] are valid. This syntax has a limitation: each 'key' has to be explicitly accessed by its original name. The normal s.key syntax doesn't provide access to the keys via variables whose values evaluate to the desired keys. An example should clarify this: Define a dictionary and initialize both with dict and k=v pairs: >>> d={'a':1,'b':2} >>> s=Struct(d,hi=10,ho=20) The return of __repr__ can be used to create a new instance: >>> s Struct({'ho': 20, 'b': 2, 'hi': 10, 'a': 1}) __str__ (called by print) shows it's not quite a regular dictionary: >>> print s Struct {a: 1, b: 2, hi: 10, ho: 20} Access by explicitly named key with dot notation: >>> s.a 1 Or like a dictionary: >>> s['a'] 1 If you want a variable to hold the key value, only dictionary access works: >>> key='hi' >>> s.key Traceback (most recent call last): File "<stdin>", line 1, in ? AttributeError: Struct instance has no attribute 'key' >>> s[key] 10 Another limitation of the s.key syntax (and Struct(key=val) initialization): keys can't be numbers. But numeric keys can be used and accessed using the dictionary syntax. Again, an example: This doesn't work: >>> s=Struct(4='hi') SyntaxError: keyword can't be an expression But this does: >>> s=Struct() >>> s[4]='hi' >>> s Struct({4: 'hi'}) >>> s[4] 'hi' """ # Attributes to which __setitem__ and __setattr__ will block access. # Note: much of this will be moot in Python 2.2 and will be done in a much # cleaner way. __protected = ('copy dict dictcopy get has_attr has_key items keys ' 'merge popitem setdefault update values ' '__make_dict __dict_invert ').split() def __init__(self,dict=None,**kw): """Initialize with a dictionary, another Struct, or by giving explicitly the list of attributes. Both can be used, but the dictionary must come first: Struct(dict), Struct(k1=v1,k2=v2) or Struct(dict,k1=v1,k2=v2). """ if dict is None: dict = {} if isinstance(dict,Struct): dict = dict.dict() elif dict and type(dict) is not types.DictType: raise TypeError,\ 'Initialize with a dictionary or key=val pairs.' dict.update(kw) # do the updating by hand to guarantee that we go through the # safety-checked __setitem__ for k,v in dict.items(): self[k] = v def __setitem__(self,key,value): """Used when struct[key] = val calls are made.""" if key in Struct.__protected: raise KeyError,'Key '+`key`+' is a protected key of class Struct.' self.__dict__[key] = value def __setattr__(self, key, value): """Used when struct.key = val calls are made.""" self.__setitem__(key,value) def __str__(self): """Gets called by print.""" return 'Struct('+str(self.__dict__)+')' def __repr__(self): """Gets called by repr. A Struct can be recreated with S_new=eval(repr(S_old)).""" return 'Struct('+str(self.__dict__)+')' def __getitem__(self,key): """Allows struct[key] access.""" return self.__dict__[key] def __contains__(self,key): """Allows use of the 'in' operator.""" return self.__dict__.has_key(key) def __iadd__(self,other): """S += S2 is a shorthand for S.merge(S2).""" self.merge(other) return self def __add__(self,other): """S + S2 -> New Struct made form S and S.merge(S2)""" Sout = self.copy() Sout.merge(other) return Sout def __sub__(self,other): """Return S1-S2, where all keys in S2 have been deleted (if present) from S1.""" Sout = self.copy() Sout -= other return Sout def __isub__(self,other): """Do in place S = S - S2, meaning all keys in S2 have been deleted (if present) from S1.""" for k in other.keys(): if self.has_key(k): del self.__dict__[k] def __make_dict(self,__loc_data__,**kw): "Helper function for update and merge. Return a dict from data." if __loc_data__ == None: dict = {} elif type(__loc_data__) is types.DictType: dict = __loc_data__ elif isinstance(__loc_data__,Struct): dict = __loc_data__.__dict__ else: raise TypeError, 'Update with a dict, a Struct or key=val pairs.' if kw: dict.update(kw) return dict def __dict_invert(self,dict): """Helper function for merge. Takes a dictionary whose values are lists and returns a dict. with the elements of each list as keys and the original keys as values.""" outdict = {} for k,lst in dict.items(): if type(lst) is types.StringType: lst = lst.split() for entry in lst: outdict[entry] = k return outdict def clear(self): """Clear all attributes.""" self.__dict__.clear() def copy(self): """Return a (shallow) copy of a Struct.""" return Struct(self.__dict__.copy()) def dict(self): """Return the Struct's dictionary.""" return self.__dict__ def dictcopy(self): """Return a (shallow) copy of the Struct's dictionary.""" return self.__dict__.copy() def popitem(self): """S.popitem() -> (k, v), remove and return some (key, value) pair as a 2-tuple; but raise KeyError if S is empty.""" return self.__dict__.popitem() def update(self,__loc_data__=None,**kw): """Update (merge) with data from another Struct or from a dictionary. Optionally, one or more key=value pairs can be given at the end for direct update.""" # The funny name __loc_data__ is to prevent a common variable name which # could be a fieled of a Struct to collide with this parameter. The problem # would arise if the function is called with a keyword with this same name # that a user means to add as a Struct field. newdict = Struct.__make_dict(self,__loc_data__,**kw) for k,v in newdict.items(): self[k] = v def merge(self,__loc_data__=None,__conflict_solve=None,**kw): """S.merge(data,conflict,k=v1,k=v2,...) -> merge data and k=v into S. This is similar to update(), but much more flexible. First, a dict is made from data+key=value pairs. When merging this dict with the Struct S, the optional dictionary 'conflict' is used to decide what to do. If conflict is not given, the default behavior is to preserve any keys with their current value (the opposite of the update method's behavior). conflict is a dictionary of binary functions which will be used to solve key conflicts. It must have the following structure: conflict == { fn1 : [Skey1,Skey2,...], fn2 :
... read more »
|
Sat, 28 May 2005 08:21:19 GMT |
|
 |
Manuel M. Garci #5 / 5
|
 half object/half dictionary
Thanks for the code! The "merge" method looks like a very good idea. Manuel
|
Sat, 28 May 2005 09:23:26 GMT |
|
|
|