Lexical scope and closures 
Author Message
 Lexical scope and closures

I read recently a flaw I did not previously know about...

"Blocks (closures) in Ruby do not introduce a new scope, only methods
do.
This is generally acknowledged as a flaw by the Ruby community but is
left
in for backwards compatibility.  For example:

x = 0
[1,2,3].each{|x| print x}
#x now equals 3"

Of course this is just wrong. I am hoping no one expects an
implementation of Ruby to actually behave this way.

Is it generally accepted that programs should not expect this? I would
not be ashamed to break this code in a new implementation of Ruby.

-Patrick



Sun, 31 Jul 2005 12:16:08 GMT  
 Lexical scope and closures
Quote:
----- Original Message -----

Newsgroups: comp.lang.ruby

Sent: Tuesday, February 11, 2003 10:21 PM
Subject: Lexical scope and closures

> I read recently a flaw I did not previously know about...

> "Blocks (closures) in Ruby do not introduce a new scope, only methods
> do.
> This is generally acknowledged as a flaw by the Ruby community but is
> left
> in for backwards compatibility.  For example:

> x = 0
> [1,2,3].each{|x| print x}
> #x now equals 3"

I sympathize with you in a way, Patrick, but I
do find your choice of language a little strong.

I'm not certain it's "generally acknowledged as
a flaw" (nor where you are quoting).

There is certainly some disagreement as to how
some issues ought to be handled. Matz himself
has said he made mistakes in this area; I'm only
willing to consider something a true design flaw
when he says it is.

> Of course this is just wrong. I am hoping no one expects an
> implementation of Ruby to actually behave this way.

It surprised me at first, but I have had more than
three years to get used to it.

Don't make the mistake of thinking that the list
between the bars is like a method's formal parameter
list. There are some similarities. But it isn't a
formal parameter list, never was, and (I'm guessing)
never will be.

For example, have you considered the fact that any
"assignable" entity can be placed there, for example,
a writable attribute?

  scrollbar = ScrollBar.new(...)
  (0..100).each {|scrollbar.percent| sleep(0.1) }

> Is it generally accepted that programs should not expect this? I would
> not be ashamed to break this code in a new implementation of Ruby.

As I said, I've gotten used to it. I do *sometimes*
depend on this behavior, usually when I want to save
a value after a loop terminates normally or abnormally.
It's a minor thing, but it saves an extra variable.

Of course, it goes without saying that however Matz
decides to reconcile all these issues, I will simply
learn it and live with it. I suppose we all shall.

Hal



Sun, 31 Jul 2005 12:52:37 GMT  
 Lexical scope and closures
Hi --

Quote:

> I read recently a flaw I did not previously know about...

> "Blocks (closures) in Ruby do not introduce a new scope, only methods
> do.
> This is generally acknowledged as a flaw by the Ruby community but is
> left
> in for backwards compatibility.  For example:

> x = 0
> [1,2,3].each{|x| print x}
> #x now equals 3"

> Of course this is just wrong. I am hoping no one expects an
> implementation of Ruby to actually behave this way.

> Is it generally accepted that programs should not expect this? I would
> not be ashamed to break this code in a new implementation of Ruby.

Let us know if you need help choosing a name for your new language :-)

David

--
David Alan Black


Web:  http://pirate.shu.edu/~blackdav



Sun, 31 Jul 2005 21:26:06 GMT  
 Lexical scope and closures

Quote:

> I read recently a flaw I did not previously know about...

> "Blocks (closures) in Ruby do not introduce a new scope, only methods
> do.

It's not actually true. Blocks do introduce a new scope, iff the variable
name has not been used already. So

  # no previous use of 'x'
  [1,2,3].each{|x| print x}

creates a local instance of 'x' for each time the block executes. This is an
important distinction where the block is being run as a thread, or is
recursive.

Quote:
> This is generally acknowledged as a flaw by the Ruby community but is
> left
> in for backwards compatibility.  For example:

> x = 0
> [1,2,3].each{|x| print x}
> #x now equals 3"

Matz is thinking of changing the behaviour for a future version of Ruby as
follows:

(1) block parameters are always locally scoped (so '|x|' in that example
will behave as if the outside 'x' did not exist, although a warning will be
printed because of the re-use of the name; but x will still be 0 after the
iterator has finished)

(2) variables introduced _within_ a block are always scoped to the entire
method, not the block. e.g.

 [1,2,3].each{|x| j = x}

will behave the same as

 j = nil
 [1,2,3].each{|x| j = x}

and hence the value of j will persist after the iterator terminates.

(3) some other syntax will be introduced for cases where you really need j
to be a block-local variable (e.g. if this block calls itself recursively or
is a thread). This might be something like:

 [1,2,3].each{|x|
   local { |j|
     j = x
   }
 }

This has been discussed at great length :-)

Regards,

Brian.



Mon, 01 Aug 2005 07:11:53 GMT  
 Lexical scope and closures

Quote:

> (3) some other syntax will be introduced for cases where you really need j
> to be a block-local variable (e.g. if this block calls itself recursively or
> is a thread). This might be something like:

>  [1,2,3].each{|x|
>    local { |j|
>      j = x
>    }
>  }

> This has been discussed at great length :-)

I know, but let me catch my chance to state that
  I want sugar for the above!
(never thought I'd ask for that at the time I admired C's terse syntax,
but I'm requesting terseness, anyway: the above involves lots of typing
and another indentation level)

--
 _           _                            
| |__   __ _| |_ ___ _ __ ___   __ _ _ __  
| '_ \ / _` | __/ __| '_ ` _ \ / _` | '_ \
| |_) | (_| | |_\__ \ | | | | | (_| | | | |
|_.__/ \__,_|\__|___/_| |_| |_|\__,_|_| |_|
        Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

Footnotes are for things you believe don't really belong in LDP manuals,
but want to include anyway.
        -- Joel N. Weber II discussing the 'make' chapter of LPG



Mon, 01 Aug 2005 07:21:35 GMT  
 Lexical scope and closures
Quote:
----- Original Message -----



Sent: Wednesday, February 12, 2003 5:11 PM
Subject: Re: Lexical scope and closures

> Matz is thinking of changing the behaviour for a future version of Ruby as
> follows:

> (1) block parameters are always locally scoped (so '|x|' in that example
> will behave as if the outside 'x' did not exist, although a warning will
be
> printed because of the re-use of the name; but x will still be 0 after the
> iterator has finished)

I didn't realize that. So we ARE breaking
old code, eh?

Hal



Mon, 01 Aug 2005 07:27:51 GMT  
 Lexical scope and closures

Quote:
> (3) some other syntax will be introduced for cases where...

God help us all. I thought Ruby was a simple language.

If a variable is only used in the scope of a block, then let the darn compiler
figure that out.

-Patrick



Mon, 01 Aug 2005 07:35:34 GMT  
 Lexical scope and closures

Quote:

> > Matz is thinking of changing the behaviour for a future version of Ruby as
> > follows:

> > (1) block parameters are always locally scoped (so '|x|' in that example
> > will behave as if the outside 'x' did not exist, although a warning will
> be
> > printed because of the re-use of the name; but x will still be 0 after the
> > iterator has finished)

> I didn't realize that. So we ARE breaking
> old code, eh?

Yes. That's one reason for the warning; the other reason is because Matz
finds the practice of 'shadowing' distasteful :-)

It will also break old code where it depends on a variable introduced in a
block being local, e.g.

   Thread.new {
     i = 0
     ... use i locally
   }

I don't remember seeing a warning for this situation being discussed, but it
would seem logical to do so.

What you gain: probably less surprising behaviour. "a = 0" always binds to
the method scope (whether or not it is inside a block), and "|a|" is always
block-local (whether or not 'a' has been used before).

Regards,

Brian.



Mon, 01 Aug 2005 07:50:04 GMT  
 Lexical scope and closures
Quote:
----- Original Message -----



Sent: Wednesday, February 12, 2003 5:50 PM
Subject: Re: Lexical scope and closures

> > I didn't realize that. So we ARE breaking
> > old code, eh?

> Yes. That's one reason for the warning; the other reason is because Matz
> finds the practice of 'shadowing' distasteful :-)

<sigh> I never really understood the term "shadowing." I thought
he meant that a local assignment inside a block resulting in a
redefinition was distasteful. Apparently that's the exact opposite
of what he meant.

What happens to the case where obj has a meth= method, and
we put obj.meth between vertical bars?

And how will I save the value of an iterated variable? Will I have
to assign it to a var in the outer scope as soon as I enter the
block?

> It will also break old code where it depends on a variable introduced in a
> block being local, e.g.

>    Thread.new {
>      i = 0
>      ... use i locally
>    }

> I don't remember seeing a warning for this situation being discussed, but
it
> would seem logical to do so.

Hmm, even more confusing. But perhaps you could simply add
your block-local variable(s) to the list:  Thread.new {|i|...

> What you gain: probably less surprising behaviour. "a = 0" always binds to
> the method scope (whether or not it is inside a block), and "|a|" is
always
> block-local (whether or not 'a' has been used before).

Less surprising, but an imperfect solution IMO.

Still, none of the solutions has seemed perfect to me.

It seems that Matz's INVIOLABLE rule is to keep Ruby
declarationless. Every solution I saw (I think) that
kept all functionality involved some kind of declaration
mechanism.

Hal



Mon, 01 Aug 2005 08:09:07 GMT  
 Lexical scope and closures
Hi,

In message "Re: Lexical scope and closures"

|What happens to the case where obj has a meth= method, and
|we put obj.meth between vertical bars?

This will cause warning first, then eventually become error in the far
future version.  Block parameters will be more like method arguments.

|And how will I save the value of an iterated variable? Will I have
|to assign it to a var in the outer scope as soon as I enter the
|block?

You have to assign to a variable from a block parameter.

|> It will also break old code where it depends on a variable introduced in a
|> block being local, e.g.
|>
|>    Thread.new {
|>      i = 0
|>      ... use i locally
|>    }
|>
|> I don't remember seeing a warning for this situation being discussed, but
|it
|> would seem logical to do so.
|
|Hmm, even more confusing. But perhaps you could simply add
|your block-local variable(s) to the list:  Thread.new {|i|...

Yes, it's possible, even now.

|It seems that Matz's INVIOLABLE rule is to keep Ruby
|declarationless. Every solution I saw (I think) that
|kept all functionality involved some kind of declaration
|mechanism.

Yes, for variable declarations.  But I'm always open for new ideas.

                                                        matz.



Mon, 01 Aug 2005 08:42:03 GMT  
 Lexical scope and closures

Quote:

>> I would
>> not be ashamed to break this code in a new implementation of Ruby.

> Let us know if you need help choosing a name for your new language :-)

Exactly. I think my second round of interest in Ruby lasted about 24
hours. 8^)

No need for new language... just back to Scheme where idealists can
live safe and sound!

-Patrick



Mon, 01 Aug 2005 09:50:47 GMT  
 Lexical scope and closures

Quote:

> >> I would
> >> not be ashamed to break this code in a new implementation of Ruby.

> > Let us know if you need help choosing a name for your new
> language :-)

> Exactly. I think my second round of interest in Ruby lasted
> about 24 hours. 8^)

Pls don't lose interest. Ruby needs more ruby hackers.. You have a point
really (in fact, Matz is working on that).

Quote:
> No need for new language... just back to Scheme where
> idealists can live safe and sound!

Ruby stands on the boundary bw the idealists and the fundamentalists. It
gives you flexible power while allowing simple natural discipline.

You are always welcome in the Ruby community. Pls just lurk at least, you
will see what I mean.. I would love to see the Ruby community grow and make
waves...

Quote:
> -Patrick

kind regards -botp


Mon, 01 Aug 2005 10:35:16 GMT  
 Lexical scope and closures

Quote:

> > (3) some other syntax will be introduced for cases where...

> God help us all. I thought Ruby was a simple language.

> If a variable is only used in the scope of a block, then let the darn compiler
> figure that out.

That modification is not meant to spoon-feed the compiler (which does
BTW not exist) but rather to _simplify_ the scoping rules. The AST
walker always knows the truth about the program; the programmer might
not. The proposed change makes it easier to realize what's going on.

--
 _           _                            
| |__   __ _| |_ ___ _ __ ___   __ _ _ __  
| '_ \ / _` | __/ __| '_ ` _ \ / _` | '_ \
| |_) | (_| | |_\__ \ | | | | | (_| | | | |
|_.__/ \__,_|\__|___/_| |_| |_|\__,_|_| |_|
        Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

#if _FP_W_TYPE_SIZE < 64
#error "Only stud muffins allowed, schmuck."
#endif
        -- linux/arch/sparc64/quad.c



Mon, 01 Aug 2005 16:44:36 GMT  
 Lexical scope and closures

Quote:


> > > (3) some other syntax will be introduced for cases where...

> > God help us all. I thought Ruby was a simple language.

> > If a variable is only used in the scope of a block, then let the darn compiler
> > figure that out.

> That modification is not meant to spoon-feed the compiler (which does
> BTW not exist) but rather to _simplify_ the scoping rules. The AST
> walker always knows the truth about the program; the programmer might
> not. The proposed change makes it easier to realize what's going on.

That's exactly it.

I think it's a barrier to newcomers that "a=0" has various different
behaviours depending on where you put it and whether or not you had a
previous assignment to "a" elsewhere in your method. I understand it now,
but it took a while.

Furthermore, your code may sometimes *rely* on variables being block-local,
but you have no way to signal your intention other than the *absence* of an
assignment outside of the block... documentation by omission :-) I end up
using bizarre names like "thread_i" to indicate this.

Essentially, we have four different behaviours now, all of which are useful,
but all of which are implied:

(1) 'a' (assigned variable) is bound to the method's local variables

  def method
    a = 0
    [1,2,3].each {|p|
      a = p             <<<
    }
    puts a
  end

(2) 'a' is local to the block

  def method
    [1,2,3].each {|p|
      a = p             <<<
    }
  end

(3) 'p' (block parameter) is bound to the method's local variables

  def method
    p = nil
    [1,2,3].each {|p|   <<<
      a = p
    }
    puts p
  end

(4) 'p' is local to the block

   def method
     [1,2,3].each {|p|  <<<
       a = p
     }
   end

And actually cases (1) and (3) have variants where they bind to the
variables of an enclosing block, as opposed to the method itself:

(1a) 'a' is bound to an enclosing block

  def method
    [1,2,3].each {|p|
      a = nil           <<< a is local to outer block
      [4,5,6].each {|q|
        a = q           <<< this is the same 'a' in outer block
      }
      puts a
    }
  end

(3a) 'q' is bound to an enclosing block

  def method
    [1,2,3].each {|p|
      q = nil
      [4,5,6].each {|q| <<< bound to 'q' in outer block
        a = q
      }
      puts q
    }
  end

These last two really do seem to be different, because a method is not the
same as a Proc object, and so a method local variable is not the same as a
block local variable.

I think all the above cases will appear in real programs, except IMO cases
(3) and (3a) are ugly hacks which nobody should be allowed to use :-) That
is, parameters to a block should be like the formal parameters to a method,
which are always local. That is acknowledged by Matz in the proposed New
Rules.

But apart from that, we still have to signal to the interpreter whether we
want 'a' or 'p' to be local or bound to enclosing scope, and whatever you do
that's going to involve syntax rules. Let me try to summarise:

Current rules
=============

'a' bound       previous assignment to 'a'
'a' local       no previous assignment to 'a' before block
'|p|' bound     previous assignment to 'p'
'|p|' local     no previous assignment to 'p' before block

Proposed 'New Rules'
====================

'a' bound       the default*
'a' local       not available, use |a| in a fake block, e.g. local {|a| ...}
'|p|' bound     not available, use '|q| p=q'
'|p|' local     the default

*if block is nested within another block, not clear whether 'a' binds to the
entire method or to the enclosing block; this may still depend on where 'a'
was previously assigned to.

Any other set of rules is going to have to have syntax for each of these
cases, and you just choose your sugar to taste. Many have been suggested.
e.g.

Everything defaults to bound to method
======================================

'a' bound       a=0
'a' local       my a=0          or      %a=0
'|p|' bound     |p|
'|p|' local     |my p|          or      |%p|

Everything defaults to block-local
==================================

'a' bound       our a=0         or      { |p| <a> ... }
'a' local       a=0
'|p|' bound     |our p|
'|p|' local     |p|

Matz doesn't want any explicit declarations, so basically the new rules
forbid two behaviours completely ('a' local and '|p|' bound), with
workarounds if you need those behaviours.

'|p|' bound is not very useful anyway, so your fundamental problem boils
down to how to choose between the two behaviours for local variables:

   max = 0
   obj.each {|i|
     max = i if i > max   # max must be BOUND or this doesn't work
   }
   puts max

   Thread.new {
     tmp = Obj.new        # tmp must be LOCAL or this doesn't work
     tmp.doit
   }

Regards,

Brian.

P.S. All this ignores the fact that "puts a" might be a method call or a
local variable depending on whether "a" has been assigned to previously. I
don't mind explaining that to nubies... it's having further levels of
complexity over assignments which is likely to put people off!



Mon, 01 Aug 2005 19:54:38 GMT  
 Lexical scope and closures

B> Essentially, we have four different behaviours now

 No, there are only 2 cases. No need to give complex explanation for
 something which is simple.

 What you call a "block parameter" don't exist, this is is *just* an
 assigned variable : understand this and you'll understand the rules used
 by ruby actually.

Guy Decoux



Mon, 01 Aug 2005 20:01:52 GMT  
 
 [ 67 post ]  Go to page: [1] [2] [3] [4] [5]

 Relevant Pages 

1. OO, lexical scope, closure, tail-recursion, continuation

2. Nested scopes and lexical closures

3. Lexical Closures in Dylan

4. Are Agents Lexical Closures?

5. Yet Another Idea for lexical closures in Python

6. Compiling lexical closures in Common Lisp

7. Lambda-expression and lexical closure

8. CL and lexical closures

9. Compiling lexical closures

10. lexical closures and python

11. Dynamic Vs Lexical Scope Rule

12. Scheme is Lisp with Lexical Scope

 

 
Powered by phpBB® Forum Software