Markus' ruby-parser (part 2) 
Author Message
 Markus' ruby-parser (part 2)

This is a follow-up to comment on some of the goofs I made in the version
I committed tonight. I made a few horrible mistakes, most of all not
running the code with the assertion-checking enabled. so the code is
still, er, problematic. Tests are passed without assertion-checking; in
the process, I have made a few little changes to parse.y (the racc one)
and ParserTests.rb.

However there are a large number of type-checking issues that have not
been resolved. I want RubySchema to be simple and easy to use. I feel that
in a lot of places, it is not.

One big reason for that is that RubySchema is currently designed to
validate the output of a ruby parser as close as Ruby's built-in parser
would (even though it horribly fails at that). I think RubySchema should
be somewhat liberalized, to accept things that ruby parsers possibly
wouldn't produce, but without making things more difficult for code
processors.

It would be futile for me at this point to just fix RubySchema according
to current rules if there's a good potential for simplifying the rules.

Note that I still would like RubySchema to be able to represent everything
that can be parsed by a Ruby parser such that it can be unparsed back to
source with as much fidelity as possible to the original code that was
parsed. The only information i want to lose is whitespace and comments and
a pair of parens here and there. I want to keep "if not" separate from
"unless" and that kind of thing.

The change I'd like to do the most is reduce the number of categories like
Statement,Expr,Arg,Primary, which are useful for the parser and mostly
nothing else. I'd like to reduce that to just Expr. It would ease things a
lot.

Another change would be that everywhere a Body is required, a Statement
would be required instead (an Expr if i make the above change).

Those are the two big changes I'd like to do ASAP. I just want some
feedback to know whether it makes sense. I want to make many more changes
and justify the way all things are done in RubySchema, which means
changing some parts in order to make them justifiable. I mean, after the
two above changes, RubySchema will still be far from polished.

So... I await your feedback with (im)patience.

________________________________________________________________
Mathieu Bouchard                   http://www.*-*-*.com/ ~matju



Fri, 02 Jul 2004 03:50:21 GMT  
 Markus' ruby-parser (part 2)

Quote:

> Note that I still would like RubySchema to be able to represent everything
> that can be parsed by a Ruby parser such that it can be unparsed back to
> source with as much fidelity as possible to the original code that was
> parsed. The only information i want to lose is whitespace and comments and
> a pair of parens here and there. I want to keep "if not" separate from
> "unless" and that kind of thing.

Would it's serialization method (offer various options for) pretty
print(ing)?

Quote:
> So... I await your feedback with (im)patience.

I'd like to use it where my program is generating Ruby code (no parsing
involved).
So it
* should be easy, intuitive and convenient to use;
* should offer a complete set of classes and methods, so that any Ruby
   program can be generated (including comments, shebangs and all);
* 's serialization method(s) should offer some control over layout:
   pretty printing; multiple files and locations; etc.

e{*filter*}d,

Tobi

--
http://www.*-*-*.com/



Fri, 02 Jul 2004 04:16:58 GMT  
 Markus' ruby-parser (part 2)

Quote:

> This is a follow-up to comment on some of the goofs I made in the version

This is another such post to comment on some other goofs, etc.

I rethought about the whole thing. It seems that for a while I forgot some
important facts about RubySchema. In particular, every evaluable form
should have its own class or module, because otherwise some hacks would be
required. It also happens that a Var is evaluable. And you see what? I
didn't want to turn all Vars into Forms, and Markus did. Well, I repaired
my mistake, it's okay now =) The dumps may just look a little more ugly
but I think that it's still a good tradeoff.

Now. I want to state that I don't see much of a point separating GVars and
SVars; I merged them all in GVars; that is, except for __FILE__ and
__LINE__, which are, er, exceptions, because they are not variables at
all, but are still considered so because I haven't considered anything
else yet. "self" has its own class "Self" as you did write; the other
pseudo-variables, nil, false, true, are part of LAtom, as was the case
from the start, and that's because they really are very much constants. Is
there anything I should change here? (remark: GVars can be made pretty
special. Ruby has its own mechanism for setting accessors on globals)

I'm testing with both 1.6 and 1.7; I just hit a segmentation fault in 1.7.
It looks like a NULL pointer in rb_mark_tbl() while marking
obj->as.object.iv_tbl in rb_gc_mark_children(), gc.c:780. There were less
than 200 stackframes.

  Revision    Path
  1.4         lib/metaruby/Contract.rb
  1.8         lib/metaruby/Type.rb
  1.11        lib/metaruby/RubySchema.rb

________________________________________________________________
Mathieu Bouchard                   http://hostname.2y.net/~matju



Sun, 04 Jul 2004 00:02:41 GMT  
 Markus' ruby-parser (part 2)

M> I'm testing with both 1.6 and 1.7; I just hit a segmentation fault in 1.7.
M> It looks like a NULL pointer in rb_mark_tbl() while marking
M> obj-> as.object.iv_tbl in rb_gc_mark_children(), gc.c:780. There were less
M> than 200 stackframes.

 Well, it's best if you give a _small_ script to reproduce the problem.

Guy Decoux



Sun, 04 Jul 2004 00:08:03 GMT  
 Markus' ruby-parser (part 2)

Quote:


> M> I'm testing with both 1.6 and 1.7; I just hit a segmentation fault in 1.7.
> M> It looks like a NULL pointer in rb_mark_tbl() while marking
> M> obj-> as.object.iv_tbl in rb_gc_mark_children(), gc.c:780. There were less
> M> than 200 stackframes.
>  Well, it's best if you give a _small_ script to reproduce the problem.

Dans un monde parfait. It's 300k of required files including a big RACC
generated parser. Most of the handwritten code hasn't been written by me.
When I try to trace the code by inserting p lines, or when running it in
gdb, the segfault happens in a different place.

I compiled with -lefence and when the program aborted, it did so while
sweep()ing a RStruct. So I fetched that one line in our code:

  Example = Struct.new :code, :ast

and i replaced it by:

  class Example<Form; fields_are \
    [:code,Object],
    [:ast,Object] end

(where Form is a pure-ruby Struct-like system)

and then it didn't crash anymore; i also tried inserting a GC.start after
each test run and it still didn't crash. That doesn't explain the iv_tbl
problem, but that may be just because of corruption caused by a bad
free().

________________________________________________________________
Mathieu Bouchard                   http://hostname.2y.net/~matju



Sun, 04 Jul 2004 00:56:04 GMT  
 Markus' ruby-parser (part 2)

M>   Example = Struct.new :code, :ast

M> and i replaced it by:

M>   class Example<Form; fields_are \
M>     [:code,Object],
M>     [:ast,Object] end

M> (where Form is a pure-ruby Struct-like system)

M> and then it didn't crash anymore;

 With a complete example I can explain this :-) You must know that Struct
 are very special (look at the source)

M> i also tried inserting a GC.start after
M> each test run and it still didn't crash.

 I can also explain this :-)) but without an example ...

Guy Decoux



Sun, 04 Jul 2004 01:01:23 GMT  
 Markus' ruby-parser (part 2)

Quote:


> I rethought about the whole thing...

Ok, now that you understand your ideas ... :-)

Do you want to be the new ruby-parser maintainer?

I'm currently spending each free minute on a Ruby VM, and I will keep doing
so, so that I can relase something (yet again pre-alpha) in a few weeks.

Quote:
> Now. I want to state that I don't see much of a point separating GVars and
> SVars; I merged them all in GVars; that is, except for __FILE__ and
> __LINE__, which are, er, exceptions, because they are not variables at
> all, but are still considered so because I haven't considered anything
> else yet. "self" has its own class "Self" as you did write; the other
> pseudo-variables, nil, false, true, are part of LAtom, as was the case
> >from the start, and that's because they really are very much constants.
> Is
> there anything I should change here? (remark: GVars can be made pretty
> special. Ruby has its own mechanism for setting accessors on globals)

You said that RubySchema should be the 'meaning-conveying parts'. `$_' and
`$some_global_var' have different meaning, especially in a multithreaded
environment. I'd prefer VarSpecial, or SVar.

When I've changed your RubySchema.rb that was because (1) matz' parse.y said
something different. Or: (2) for constructing the Nodes in the action
snippets. In parse.y there are a few places where arguments get passed from action
to action as arrays. As far as I remember, I did some `extensions' to
RubySchema to allow this.

Removing Statment,Expr,Arg,Literal

Would be absolutely no problem for me.

`%c{}' `%u{}'

I dislike the syntax; (even `%w'; it's handy, but not nice;) looks too much
like Perl. Couldn't one borrow the syntax from Lisp Macros? How could one do
that?
Somebody here knows Forth' parsing words? Could one build an extensible Ruby
parser based on that idea?

Using % to mean 'quote' is ok, but then the single character... w does not
really imply word.

cheers,
markus

--
-------------------------------------------------



Sun, 04 Jul 2004 05:45:05 GMT  
 Markus' ruby-parser (part 2)

Quote:


> Do you want to be the new ruby-parser maintainer?

Oh boy. I actually waited half a year for someone else to do the work
because I didn't want to mess with parsers. I guess I can do some
maintenance on it (getting cvs commit perms)...

Quote:
> I'm currently spending each free minute on a Ruby VM, and I will keep doing
> so, so that I can relase something (yet again pre-alpha) in a few weeks.

Can you give me more information on that Ruby VM ? is it Robert's project
or your own project ? What are its main design principles ? (didn't we
talk about this particular project in the past?)

Quote:
> You said that RubySchema should be the 'meaning-conveying parts'. `$_' and
> `$some_global_var' have different meaning, especially in a multithreaded
> environment. I'd prefer VarSpecial, or SVar.

Global vars are very much *special* vars; they're the only real variables
that can be traced (Kernel.#trace_var). I guess the $ can be considered as
a slashed S where S stands for special. I want you to note that although
many $-vars with non-text names have special scoping, many others don't,
and in addition some $-vars with text names have special scoping
(e.g. $SAFE). I'm not too sure I want to separate those cases in the
parser. Maybe it would become like that one day, but for now I'd prefer to
keep them together.

BTW is it normal that "alias" on global vars appears exactly the same as
assignment, except with the more special vars?

$foo=42
$bar=210
alias $foo $bar
$foo #==> 210
$foo = :hello
$bar #==> 210

alias $foo $_
$foo #==> nil
$_ = 42
$foo #==> 42
$foo = :blah
$_ #==> :blah

Quote:
> Removing Statment,Expr,Arg,Literal
> Would be absolutely no problem for me.

thanks for your opinion.

Quote:
> `%c{}' `%u{}'
> I dislike the syntax; (even `%w'; it's handy, but not nice;) looks too
> much like Perl.

"Looks too much like Perl" is a real bad excuse and I don't want to hear
it.

Quote:
> Couldn't one borrow the syntax from Lisp Macros?

You really don't want to do that. Even though Lisp list-interpolation does
it more nicely than my Perl-looking constructs, they rely on the fact that
Lisp doesn't have many special characters; Ruby is full of special
characters. You know what the Lisp syntax is for that? Backtick for a
semi-quote (that's %c{}), comma for an interpolation (that's %u{}).

Comma! is there any other Ruby char less overloadable than the comma?

Quote:
> Using % to mean 'quote' is ok, but then the single character... w does not
> really imply word.

Please explain.

________________________________________________________________
Mathieu Bouchard                   http://hostname.2y.net/~matju



Tue, 06 Jul 2004 05:30:53 GMT  
 Markus' ruby-parser (part 2)
    Are there any links or help on how to use the DirectX libraries from
Ruby ? Is it a matter of using the COM interfaces or would I need to use
SWIG to somehow wrap the libraries ? Any help is appreciated.

Thanks

Quote:
----- Original Message -----





Sent: Thursday, January 17, 2002 2:23 PM
Subject: Re: this ruby-parser



> > Do you want to be the new ruby-parser maintainer?

> Oh boy. I actually waited half a year for someone else to do the work
> because I didn't want to mess with parsers. I guess I can do some
> maintenance on it (getting cvs commit perms)...

> > I'm currently spending each free minute on a Ruby VM, and I will keep
doing
> > so, so that I can relase something (yet again pre-alpha) in a few weeks.

> Can you give me more information on that Ruby VM ? is it Robert's project
> or your own project ? What are its main design principles ? (didn't we
> talk about this particular project in the past?)

> > You said that RubySchema should be the 'meaning-conveying parts'. `$_'
and
> > `$some_global_var' have different meaning, especially in a multithreaded
> > environment. I'd prefer VarSpecial, or SVar.

> Global vars are very much *special* vars; they're the only real variables
> that can be traced (Kernel.#trace_var). I guess the $ can be considered as
> a slashed S where S stands for special. I want you to note that although
> many $-vars with non-text names have special scoping, many others don't,
> and in addition some $-vars with text names have special scoping
> (e.g. $SAFE). I'm not too sure I want to separate those cases in the
> parser. Maybe it would become like that one day, but for now I'd prefer to
> keep them together.

> BTW is it normal that "alias" on global vars appears exactly the same as
> assignment, except with the more special vars?

> $foo=42
> $bar=210
> alias $foo $bar
> $foo #==> 210
> $foo = :hello
> $bar #==> 210

> alias $foo $_
> $foo #==> nil
> $_ = 42
> $foo #==> 42
> $foo = :blah
> $_ #==> :blah

> > Removing Statment,Expr,Arg,Literal
> > Would be absolutely no problem for me.

> thanks for your opinion.

> > `%c{}' `%u{}'

> > I dislike the syntax; (even `%w'; it's handy, but not nice;) looks too
> > much like Perl.

> "Looks too much like Perl" is a real bad excuse and I don't want to hear
> it.

> > Couldn't one borrow the syntax from Lisp Macros?

> You really don't want to do that. Even though Lisp list-interpolation does
> it more nicely than my Perl-looking constructs, they rely on the fact that
> Lisp doesn't have many special characters; Ruby is full of special
> characters. You know what the Lisp syntax is for that? Backtick for a
> semi-quote (that's %c{}), comma for an interpolation (that's %u{}).

> Comma! is there any other Ruby char less overloadable than the comma?

> > Using % to mean 'quote' is ok, but then the single character... w does
not
> > really imply word.

> Please explain.

> ________________________________________________________________
> Mathieu Bouchard                   http://hostname.2y.net/~matju



Tue, 06 Jul 2004 05:36:16 GMT  
 Markus' ruby-parser (part 2)

Quote:
>     Are there any links or help on how to use the DirectX libraries from
> Ruby ? Is it a matter of using the COM interfaces or would I need to use
> SWIG to somehow wrap the libraries ? Any help is appreciated.

If you specifically want to write Ruby code using a DirectX-like programming
interface I guess you'd want to use the COM interfaces or something; I won't
even go there. But if you just need a means of accessing the DirectX
functionality (and don't mind programming against a different API) you might
check out the two SDL wrappers for Ruby that are currently listed in the
Ruby Application Archive: RUDL (http://froukepc.dhs.org/rudl/index.html) and
Ruby/SDL (http://www.kmc.gr.jp/~ohai/index.en.html).

Since these are interfaces to SDL, and the Windows implementation of SDL
uses DirectX under the hood (right?) this may be the quickest way to get
there.

Hope this helps,

Lyle



Tue, 06 Jul 2004 07:07:50 GMT  
 
 [ 13 post ] 

 Relevant Pages 

1. Markus' ruby-parser

2. Markus Dahm's OO Forth

3. Ruby Syntax Highlighting (and a Ruby Parser BUG)

4. Ruby parsers in Ruby

5. Embedded Ruby (PART II)- Ruby hackers help wanted :)

6. Ruby Tuesday - Sydney Ruby Users' Group Meeting

7. Write Ruby's Array in Ruby

8. Ruby/Glade doesn't work with Ruby 1.6.1

9. Not seeing today's ruby-talk ML items on comp.lang.ruby

10. 320x200 picture display MARKUS

11. 'pstruct' structure parser and python

12. mx.DateTime.Parser.DateFromString('crap')

 

 
Powered by phpBB® Forum Software