patch to allow dump/load to work on DATA objects 
Author Message
 patch to allow dump/load to work on DATA objects

I don't think there's any *general* way to marshal objects that have a C
struct attached to them (i.e., created with DATA_MAKE_STRUCT or
DATA_WRAP_STRUCT) and which contain arbitrary data (including references
to Ruby objects), so here's a patch that cleanly adds that
functionality.

This patch is conservative in the sense that it only affects execution
paths that resulted in exceptions before. It should not break existing
code or formatted data. The patch is based on 1.6.6.

USAGE
=====

The following methods should be implemented in a class used to wrap C
data if some of that C data needs to be serialized:

Instance methods:

_dump_data

  Returns an Object that encapsulates the data stored in the struct.

_load_data Object

  Called on the object after it is allocated, but before instance vars
are restored. (Unlike the _load method used with Marshal, this is an
instance method.) Used to populate the struct with data from the
argument object.

These two instance methods can be written in Ruby if there are accessor
methods available for reading and writing all the persistent C data. (It
may be advantageous to write them in C, though.)

Class method:

_alloc

  Invoke DATA_MAKE_STRUCT, and return the resulting object.

Notes:

(1) I originally thought of using just a string argument in place of the
Object, for consistency with _dump/_load, but that made life hard within
the _dump_data/_load_data implementation, and also it is impossible to
connect to Marhsal's hash of objects that it has seen, so your object
graph gets turned into a tree with duplicates.

(2) There is no limit parameter to _dump_data because this gets applied
automatically in the w_obejct call in the T_DATA case.

(3) The reason I didn't just use _dump/_load or redefine
Marshal.dump/load, is that my C data is very general. It may refer to
other Ruby objects. To maintain referential integrity, I need to
continue with the same Marshal.dump/load call without starting the
process from scratch with a new arg->data hash.

(4) The _dump_data/_load_data methods are called only in the case of a
T_DATA object. They are used in conjuntion with w_object/r_object.

(5) I have tested this code in my CGen/CShadow library to make sure that
both Ruby attrs and C data are marshalled, and that referential
integrity is preserved (including cycles).

(6) I've also tested that the proc argument to Marshal.load is called.
When this proc is called, instance vars are all nil, but that's the way
it works in other cases now, anyway.

<plug>
This new functionality supports CGen/CShadow, which can now manage the
marshalling of C attributes for you. I'll upload the latest version in a
day or two.
</plug>

--
Joel VanderWerf                          California PATH, UC Berkeley

http://www.*-*-*.com/                       FAX (510) 231-9512

[ marshal.patch 2K ]
--- marshal.c.orig      Mon Feb 18 21:54:34 2002

 #define TYPE_UCLASS    'C'
 #define TYPE_OBJECT    'o'
+#define TYPE_DATA       'd'
 #define TYPE_USERDEF   'u'

 static ID s_dump, s_load;
+static ID s_dump_data, s_load_data, s_alloc;


            break;

+          case T_DATA:
+            w_byte(TYPE_DATA, arg);
+            {
+                VALUE klass = CLASS_OF(obj);
+                char *path;
+
+                if (FL_TEST(klass, FL_SINGLETON)) {
+                    if (RCLASS(klass)->m_tbl->num_entries > 0 ||
+                        RCLASS(klass)->iv_tbl->num_entries > 1) {
+                        rb_raise(rb_eTypeError, "singleton can't be dumped");
+                    }
+                }
+                path = rb_class2name(klass);
+                w_unique(path, arg);
+            }
+            {
+                VALUE v;
+
+                if (!rb_respond_to(obj, s_dump_data)) {
+                    rb_raise(rb_eTypeError,
+                             "class %s needs to have instance method `_dump_data'",
+                             rb_class2name(CLASS_OF(obj)));
+                }
+                v = rb_funcall(obj, s_dump_data, 0);
+                w_object(v, arg, limit);
+            }
+            break;
+
          default:

        break;

+      case TYPE_DATA:
+        {
+            VALUE klass;
+
+            klass = rb_path2class(r_unique(arg));
+            if (!rb_respond_to(klass, s_alloc)) {
+                rb_raise(rb_eTypeError,
+                         "class %s needs to have class method `_alloc'",
+                         rb_class2name(klass));
+            }
+            v = rb_funcall(klass, s_alloc, 0);
+            if (TYPE(v) != T_DATA) {
+                rb_raise(rb_eArgError, "dump format error");
+            }
+            r_regist(v, arg);
+            if (!rb_respond_to(v, s_load_data)) {
+                rb_raise(rb_eTypeError,
+                         "class %s needs to have instance method `_load_data'",
+                         rb_class2name(klass));
+            }
+            rb_funcall(v, s_load_data, 1, r_object(arg));
+            return v;
+        }
+        break;
+
       case TYPE_MODULE_OLD:

     s_dump = rb_intern("_dump");
     s_load = rb_intern("_load");
+    s_dump_data = rb_intern("_dump_data");
+    s_load_data = rb_intern("_load_data");
+    s_alloc = rb_intern("_alloc");
+
     rb_define_module_function(rb_mMarshal, "dump", marshal_dump, -1);
     rb_define_module_function(rb_mMarshal, "load", marshal_load, -1);



Sun, 08 Aug 2004 11:57:04 GMT  
 patch to allow dump/load to work on DATA objects
Hi,

In message "patch to allow dump/load to work on DATA objects"

|I don't think there's any *general* way to marshal objects that have a C
|struct attached to them (i.e., created with DATA_MAKE_STRUCT or
|DATA_WRAP_STRUCT) and which contain arbitrary data (including references
|to Ruby objects), so here's a patch that cleanly adds that
|functionality.

Thank you for interesting idea.  I'm thinking of a new marshal
framework that is highly customizable via Serializer objects.  I think
I'm going to fix this "problem" too in that framework.

The framework should:

  * separate object traversal and object serialization.
  * object serialization scheme can be defined by users by defining
    Dumper/Loader class pair.

But it still is a vague idea.  I have too much vapor ideas. ;-)

                                                        matz.



Sun, 08 Aug 2004 12:56:01 GMT  
 patch to allow dump/load to work on DATA objects

Quote:

> Hi,

> In message "patch to allow dump/load to work on DATA objects"

> |I don't think there's any *general* way to marshal objects that have a C
> |struct attached to them (i.e., created with DATA_MAKE_STRUCT or
> |DATA_WRAP_STRUCT) and which contain arbitrary data (including references
> |to Ruby objects), so here's a patch that cleanly adds that
> |functionality.

> Thank you for interesting idea.  I'm thinking of a new marshal
> framework that is highly customizable via Serializer objects.  I think
> I'm going to fix this "problem" too in that framework.

> The framework should:

>   * separate object traversal and object serialization.
>   * object serialization scheme can be defined by users by defining
>     Dumper/Loader class pair.

That sounds like the best approach for the future. I especially like
factoring out traversal, which would be useful for other things (deep
copy/compare without serialization). But for now, what do you think
about the patch? Would you like me to send you some test code?

Quote:
> But it still is a vague idea.  I have too much vapor ideas. ;-)

>                                                         matz.

Better than no ideas...


Mon, 09 Aug 2004 01:34:05 GMT  
 patch to allow dump/load to work on DATA objects
Hi,

In message "Re: patch to allow dump/load to work on DATA objects"

|But for now, what do you think
|about the patch? Would you like me to send you some test code?

If you can live with the following:

  * it will be remain undocumented.
  * it will be unavailable in the future.

                                                        matz.



Mon, 09 Aug 2004 03:32:27 GMT  
 patch to allow dump/load to work on DATA objects

Quote:

> Hi,

> In message "Re: patch to allow dump/load to work on DATA objects"

> |But for now, what do you think
> |about the patch? Would you like me to send you some test code?

> If you can live with the following:

>   * it will be remain undocumented.
>   * it will be unavailable in the future.

>                                                         matz.

That's great. Thank you very much.

When you release the new Marshal framework, I will be happy to use that
instead.



Mon, 09 Aug 2004 05:34:52 GMT  
 patch to allow dump/load to work on DATA objects
Hi,

In message "Re: patch to allow dump/load to work on DATA objects"

|> If you can live with the following:
|>
|>   * it will be remain undocumented.
|>   * it will be unavailable in the future.
|
|That's great. Thank you very much.

OK, I start checking on your patch.

                                                        matz.



Mon, 09 Aug 2004 15:00:39 GMT  
 patch to allow dump/load to work on DATA objects
Hi,

At Thu, 21 Feb 2002 15:57:06 +0900,

Quote:

> |> If you can live with the following:
> |>
> |>   * it will be remain undocumented.
> |>   * it will be unavailable in the future.
> |
> |That's great. Thank you very much.

> OK, I start checking on your patch.

I guess r_object() should use `allocate' instead of `_alloc'
for compatibility with 1.7.

--
Nobu Nakada



Mon, 09 Aug 2004 17:08:38 GMT  
 
 [ 7 post ] 

 Relevant Pages 

1. ANN: experimental patch to allow importing from file-like objects

2. experimental patch to allow importing from file-like objects

3. dump/load a class that has C and Ruby data

4. Dumping and loading object information

5. why the auto load function don't work when I load a new library

6. FreeRIDE 0.5.0rc1: patch to allow per-user properties

7. loading javascript arrays (or listboxes) with mysql data after a page has loaded without a refresh

8. Patch allowing filechooser to select multiple files

9. Patch to Tk 3.6 listbox to allow non-contiguous selections

10. Tcl8.0b1 patch to allow widget creation in namespaces

11. PATCH: Tk4.0b1 canvas core dump

12. Calling Marshal.dump/load from C

 

 
Powered by phpBB® Forum Software